This is some code I wrote to scrape stock prices with Python. I wrote it on Jupyter notebook.<\/p>\n
First off you’ll need chromedriver (Google “download chromedriver” and get the file on the first link. Put it in the folder with your Jupyter notebook.<\/p>\n
Next, you’ll need a bunch of libraries, some of which will need to be pip installed.<\/p>\n
from selenium import webdriver<\/em> In the code below, you won’t need all of this, but I’m just copying the entire import section of my code.<\/p>\n Next, we’ll fire up a browser.<\/p>\n driverspy = webdriver.Chrome()<\/em> This should open a python controlled browser that surfs its way to Yahoo Finance and loads up the page for SPY (a popular S&P 500 ETF).<\/p>\n Finally, we’ll define a function to scrape the price and then scrape the price off this page.<\/p>\n sourcespy = driverspy.page_source<\/em> If you look at the html code of the page_source of the Yahoo page with the SPY data, you’ll see it has, buried in it, something that looks like this:<\/p>\n <span class=”Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)” data-reactid=“35”>283.82<\/span><\/span><\/strong><span class=”Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataGreen)” data-reactid=”36″>+1.72 (+0.61%)<\/span><div<\/p>\n We rely on the bolded<\/strong> part always being the same (“35”> … <\/span> and encapsulating the\u00a0bold+underlined\u00a0<\/strong><\/span>price (283.82) to extract the price. The \\d+.\\d+ tells Python to look for a positive number, a period and another positive number.<\/p>\n Now, we have a basic scraper to get prices from Yahoo finance. If we set up a loop, we can get prices every few minutes and generate a time series dataset.<\/p>\n","protected":false},"excerpt":{"rendered":" This is some code I wrote to scrape stock prices with Python. I wrote it on Jupyter notebook. First off you’ll need chromedriver (Google “download chromedriver” and get the file on the first link. Put it in the folder with your Jupyter notebook. Next, you’ll need a bunch of libraries, some of which will need […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[9,10],"tags":[],"_links":{"self":[{"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/posts\/216"}],"collection":[{"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/comments?post=216"}],"version-history":[{"count":4,"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/posts\/216\/revisions"}],"predecessor-version":[{"id":220,"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/posts\/216\/revisions\/220"}],"wp:attachment":[{"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/media?parent=216"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/categories?post=216"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.sugata.in\/index.php\/wp-json\/wp\/v2\/tags?post=216"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nimport datetime<\/em>
\nimport time<\/em>
\nfrom multiprocessing import Pool,TimeoutError<\/em>
\nimport urllib.request<\/em>
\nimport re<\/em>
\nfrom urllib.error import URLError, HTTPError<\/em><\/p>\n
\ndriverspy.get(‘https:\/\/finance.yahoo.com\/quote\/SPY?p=SPY’)<\/em><\/p>\n
\nfound = re.search(‘”35″>(\\d+\\.\\d+)<\/span>’, sourcespy).group(1<\/em>)<\/p>\n