Open Source – Parse BaaS Data Exporter
July 27th 2015
Expa is a startup studio, a company that creates new companies. We have lots of different kinds of tech op’s challenges as our portfolio companies need different kinds of support.
One ongoing challenge is how to store and extract data in a way that is efficient and automated. To sum it up, our portfolio works with disparate data stores, and more specifically than that, we don’t dictate how our portfolio companies store their data, so they are free to choose whatever product makes sense for their team. As an example, one of our portfolio companies was primarily client focused, with iOS engineers, and wasn’t ready to invest in a custom backend. That’s where Parse fits in.
Extracting data: the challenge
Parse has its own analytics platform, but their platform did not particularly meet all of our business objectives. We needed specifically to get our portfolio company’s data out of Parse and into Expa’s analytics data warehouse. However, Parse doesn’t provide a programmatic way to export your data. There have been a few approaches but no one way has emerged. One-Click Export is the only official method but it’s manual and not reasonable to automate.
The solution we came up with is just a simple command line utility – you give it your Parse credentials and a list of Parse objects that you want to export, and it will then hit Parse’s API and download the data into individual files.
Things to note
- This tool does query the Parse REST API and will (slightly) eat into your requests/second limit
- For request efficiency, we use the max allowed chunk size of 1000 records per request
- Pricing reference
Check it out
Feel free to fork to our Parse data exporter (in Python) and create a pull request to the original repo to modify and make the code better!
–Director, Technical Operations