Patrick Paul is a senior at Michigan State University studying International Relations.
| Welcome to my website! My primary interest in keeping a personal website is to organize my life--and not to woo strangers on the internet with uninformed rants on current events(sorry!). All posts will be related to any of my on-going projects, goals, gigs, and ideas. You can navigate my posts chronologically using the Archive, or by using the very cool Progress page./s/ Patrick |
The end (of the first phase) is in sight. I started this project very myopic and wrote code with very small, short-term goals in mind, but I find great joy this morning in having the rest of the code conceived in my mind (and on paper) (and on MS Word). I know exactly what is left to be implemented — everything is sure.
To date, Midas Data Miner coding efforts have focused on mining & storing raw data which I have aptly stored in the binary tree named “primary”.
I remain to be pleased with my ZODB implementation.
Since my last post, I have successfully downloaded all the market data I could find off of Yahoo! Finance. The program downloads all the market data for a specified ticker off of Yahoo’s servers, and then this information is saved into a binary tree in my ZODB, which is keyed by the tuple (date,ticker).
The download took 7 hours, and comprised 8 years of market data for all my tickers.
I’m not sure if everything is packed optimally or not, but the resulting ZODB pickle file is sixteen gigabytes.