Buy Me a Coffee? Your support is much appreciated!
PayPal Me: https://www.paypal.me/jiejenn/5
Venmo: @Jie-Jenn
Source Code:
import requests from bs4 import BeautifulSoup # pip install beautlfulsoup4 import pandas as pd # pip install pandas import win32com.client as win32 df = pd.DataFrame({'A': [10, 20], 'B': [39, 49]}) df.size tickers = ['tsla', 'fb', 'goog', 'crm'] headers= {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0'} xlApp = win32.Dispatch('Excel.Application') xlApp.Visible = True wb = xlApp.Workbooks.Add() for ticker in tickers: url = "https://www.marketwatch.com/investing/stock/{0}/company-profile".format(ticker) response = requests.get(url, headers=headers) soup = BeautifulSoup(response.content, 'html.parser') profile_info = {} element_tables = soup.select("div[class='element element--table']") for element_table in element_tables: valuation_type = element_table.h2.text.strip() df = pd.read_html(str(element_table))[0] profile_info[valuation_type] = df """ Export data to Excel spreadsheets """ ws = wb.Worksheets.Add() ws.name = ticker row_spacing = 2 for table in profile_info.items(): lastrow = ws.Cells(ws.rows.count, 1).End(-4162).row ws.cells(lastrow+row_spacing, 1).value = table[0] ws.cells(lastrow+row_spacing, 1).font.bold = True ws.Range( ws.cells(lastrow+row_spacing+1, 1), ws.cells(lastrow+table[1].shape[0]+row_spacing, table[1].shape[1]) ).value = table[1].values ws.Rows('1:' + str(row_spacing)).delete ws.columns(1).columnwidth = 30 ws.columns(2).columnwidth = 15
Hi Jie Jenn,
Thank you for uploading this video. I am a beginner in scraping. When I run this code till profile_info[valuation_type] = df, I receive “value error” saying no tables found matching pattern. Could you please tell me what I am doing wrong?
Thank you.