wsdookadr

Make Ubuntu packages 90% faster by rebuilding them

TL;DR

You can take the same source code package that Ubuntu uses to build jq, compile it again, and realize 90% better performance.

Setting

I use jq for processing GeoJSON files and other open data offered in JSON format. Today I am working with a 500MB GeoJSON file that contains the Alameda County Assessor's parcel map. I want to run a query that prints the city for every parcel worth more than a threshold amount. The program is

wsdookadr

#!/usr/bin/env python
"""
Upstream: https://gist.github.com/kylemanna/5692543
Blog post: https://blog.kylemanna.com/linux/parse-emmc-extended-csd-ecsd-registers-with-python/

Description: Get eMMC health information
Tested on ASUS T100TA
Example:

    root@fedora:/home/user# ./emmc_health.py  | rg -i 'LIFE|HEALTH|EOL'

wsdookadr

function scroll_to(splash, x, y)
  local js = string.format(
    "window.scrollTo(%s, %s);", 
    tonumber(x), 
    tonumber(y)
  )
  return splash:runjs(js)
end

wsdookadr

curl -i https://api.github.com/repos/scrooloose/nerdtree/forks |grep -e "git_url" |awk '{gsub(/,/,"");split($2,a,"/"); system("mkdir "a[4]"; cd "a[4]"; git clone " $2);}'

wsdookadr

** Find commmonly accessed tables and their use of indexes:

SELECT relname,seq_tup_read,idx_tup_fetch,cast(idx_tup_fetch AS numeric) / (idx_tup_fetch + seq_tup_read) AS idx_tup_pct FROM pg_stat_user_tables WHERE (idx_tup_fetch + seq_tup_read)>0 ORDER BY idx_tup_pct;

Returns output like:

       relname        | seq_tup_read | idx_tup_fetch |      idx_tup_pct       
----------------------+--------------+---------------+------------------------
 schema_migrations    |          817 |             0 | 0.00000000000000000000
 user_device_photos   |          349 |             0 | 0.00000000000000000000

wsdookadr

#!/usr/bin/env bash

rep() {
  i=$1
  data=$2
  ## run the replicate ....
}

# make the files
START=$(mktemp -t start-XXXX) ## signals the workers are starting

wsdookadr

CREATE OR REPLACE FUNCTION public.json_append(data json, insert_data json)
RETURNS json
IMMUTABLE
LANGUAGE sql
AS $$
    SELECT ('{'||string_agg(to_json(key)||':'||value, ',')||'}')::json
    FROM (
        SELECT * FROM json_each(data)
        UNION ALL
        SELECT * FROM json_each(insert_data)

wsdookadr

echo 'export PATH=$HOME/local/bin:$PATH' >> ~/.bashrc
. ~/.bashrc
mkdir ~/local
mkdir ~/node-latest-install
cd ~/node-latest-install
curl http://nodejs.org/dist/node-latest.tar.gz | tar xz --strip-components=1
./configure --prefix=~/local
make install # ok, fine, this step probably takes more than 30 seconds...
curl http://npmjs.org/install.sh | sh

wsdookadr

echo 'export PATH=$HOME/local/bin:$PATH' >> ~/.bashrc
. ~/.bashrc
mkdir ~/local
mkdir ~/node-latest-install
cd ~/node-latest-install
curl http://nodejs.org/dist/node-latest.tar.gz | tar xz --strip-components=1
./configure --prefix=~/local
make install # ok, fine, this step probably takes more than 30 seconds...
curl http://npmjs.org/install.sh | sh

wsdookadr

var http = require("http")

http.createServer(function (req, res) {
  // let it time out
  // just hold the connection open forever
}).listen(8080)

var r = http.createClient(8080, "localhost").request("GET", "/", {})
r.on("response", function (res) {
  console.log("got response")