The upcoming series will focus on web scraping through progressively complex case studies. Initial discussions will cover how browsers retrieve web content and the importance of HTTP headers. The author will demonstrate using various Python libraries to scrape data, with an emphasis on understanding URL structures and parsing content efficiently.
On Web Scraping
Web scraping is the process of programmatically extracting content from websites and has various applications, such as analyzing medical payments to doctors or gathering job postings. However, it raises ethical and legal questions, particularly regarding terms of use violations. The practice combines programming skills with problem-solving, creating a unique intellectual challenge.
Tracking Personal Finances
The author meticulously tracks personal finances for their family using a comprehensive Excel spreadsheet, developed over eight years. They prefer this method over software like Quicken and Mint due to issues with account synchronization. The spreadsheet categorizes transactions, aggregates balances, and includes investment tracking, with aspirations for automation.
ProjectSHERPA: a startup retrospective
Over several years, the founders of ProjectSHERPA developed a tech recruiting platform to improve the hiring process for entry-level talent. Despite innovative features and multiple pivots, including an applicant tracking system and a concierge service, they ultimately ceased operations in 2015 due to a lack of user traction and effective marketing strategies.
Better Babies
The author reflects on their early childhood experiences with flash cards based on the Doman method, which promotes early brain stimulation for newborns. Despite criticism of the method, the author shares anecdotes of impressive feats achieved during infancy. They acknowledge the lack of scientific data supporting Doman’s approach but find value in engaging activities for babies.
Can Yelp Reviews Predict Real Estate Prices?
The MBA group project focused on analyzing NYC real estate prices using various data sources, including Yelp and Trulia. Initially aiming to identify unexpected deal locations, the team shifted to exploring how Yelp reviews might predict listing prices. Ultimately, they concluded that current prices serve as the best predictor for future trends.
I Type Faster Than You
The author types around 130-140 words per minute (WPM), occasionally reaching 160 WPM, attributing speed to musical training and video games. Faster typers often use alternative keyboards like Dvorak, while the QWERTY layout was designed to slow typists. The author notes improvements in speed and accuracy, even while typing one-handed.
Don’t Look at My Baby
The author reflects on their dilemma regarding sharing pictures of their newborn daughter, Riley, amidst rising requests from family and friends. They advocate for the baby’s right to privacy and the importance of controlling one’s digital identity. To balance this, they created a password-protected site for selective photo sharing.
A Side Project
Over the past two years, the author struggled to post regularly but aims to share more insights. They volunteered for Portfolios with Purpose, aiding in stock-picking competitions, where participants invest and donate to charity. The author resolved complex market data challenges, addressing symbology issues across different exchanges to ensure accuracy in stock trading.
Plumbing and Coding
The author reflects on the complexities of plumbing while attempting to connect a water line from a sink to a fridge. Despite the seemingly simple task, various materials, sizes, and connections complicate the process. Drawing parallels to software development, the author emphasizes the necessity of understanding frameworks and gaining experience to effectively solve problems.