CSV Imports into Google Cloud BigQuery

If you have a large Postgres database, consider trying Google Cloud BigQuery for OLAP-style reporting queries. database> COPY (<query_to_denomralize_report_data>) TO '<dest_filename>' DELIMITER ',' CSV HEADER; Then upload it to GCS: $ gsutil cp <dest_filename> gs://<dest_bucket>/<dest_filename> Then, follow Loading Data into BigQuery (with Google Cloud Storage). For this to work, you’ll have to define the schema and skip the first leading row. My most recent load job took 4 seconds, loading a very large CSV file....

February 11, 2019 · itsahill00

Required Reading: The Datacenter as a Computer

Several Google Employees published The Datacenter as a Computer. It’s freely available. The text covers a broad area, from monitoring to cost modelling for power but it’s quite digestable. At a light ~130 pages it’s an easy read. The bibliography is worth digging into if you want deeper dives on specific topics.

April 3, 2014 · itsahill00