vanga’s gists

vanga / nfhs5-descriptions.json

Created September 25, 2023 16:32

NFHS5 variable descriptions

	{
	"RECH0": {
	"HHID": "Case identification uniquely identifies each household. In most surveys, this is constructed by concatenating the cluster or sample point number and the household number, but in some surveys this may be the questionnaire number taken from the front page of the questionnaire.",
	"HV000": "Alphabetic country code to identify the survey from which the data were collected. The code is based on an international standard code. This variable is 3 characters in length, with the third character indicating the format of the recode file used for this survey. For all surveys in DHS VII following this standard, this code will be 7. For example: DR7 is the Dominican Republic, HT7 is Haiti and KH7 is Cambodia.",
	"HV001": "Cluster number is the number identifying the sample point as used during the fieldwork. This variable may be a composite of several variables in the questionnaire. If so, the non- standard variables are included in RECH3 as country-specific variables.",

vanga / nfhs4-descriptions.json

Last active September 25, 2023 16:33

NFHS4 variable descriptions

	{
	"RECH0": {
	"HHID": "Case identification uniquely identifies each household. In most surveys this is constructed by concatenating the cluster or sample point number and the household number, but in some surveys this may be the questionnaire number taken from the front page of the questionnaire.",
	"HV000": "Alphabetic country code to identify the survey from which the data were collected. The code is based on an international standard code. This variable is 3 characters in length, with the third character indicating the format of the recode file used for this survey. For all surveys in DHS VI following this standard, this code will be 6. For example: DR6 is the Dominican Republic, HT6 is Haiti and KH6 is Cambodia.",
	"HV001": "Cluster number is the number identifying the sample point as used during the fieldwork. This variable may be a composite of several variables in the questionnaire. If so, the non- standard variables are included in RECH3 as country-specific variables.",

vanga / nco-occupation-codes.csv

Last active May 4, 2023 14:05

2 digit NCO 2004 Occupation codes CSV/Excel

	nco_code	label
	11	LEGISLATORS AND SENIOR OFFICIALS
	12	CORPORATE MANAGERS
	13	GENERAL MANAGERS
	21	PHYSICAL, MATHEMATICAL AND ENGINEERING SCIENCE PROFESSIONALS
	22	LIFE SCIENCE AND HEALTH PROFESSIONALS
	23	TEACHING PROFESSIONALS
	24	OTHER PROFESSIONALS
	31	PHYSICAL AND ENGINEERING SCIENCE ASSOCIATE PROFESSIONALS.
	32	LIFE SCIENCE AND HEALTH ASSOCIATE PROFESSIONALS

vanga / nic-industry-codes.csv

Created May 4, 2023 10:00

NIC 2008 Industry codes

	nic_2008_code	label
	01	Crop and animal production, hunting and related service activities
	02	Forestry and logging
	03	Fishing and aquaculture
	05	Mining of coal and lignite
	06	Extraction of crude petroleum and natural gas
	07	Mining of metal ores
	08	Other mining and quarrying
	09	Mining support service activities
	10	Manufacture of food products

vanga / se-data-processor.py

Last active September 23, 2022 13:08

For processing SE data dumps

	# Convert xml to json/parquet and stores them in TEMP_DIR
	# Column names are modified to remove "@"" at the beginning for convenience
	# Transforms tags column of posts into a list
	# Uses pandas joins and groupby features do one-one and one-many joins


	import xmltodict
	import json
	import simplejson
	import os

vanga / sample-denormalized-question.json

Created September 19, 2022 00:54

	{
	"Id": 1,
	"PostTypeId": 1,
	"AcceptedAnswerId": 52,
	"CreationDate": "2017-02-28T16:47:57.887",
	"Score": 19,
	"ViewCount": 407,
	"Body": "<p><a href=\"https://jenkins.io/\" rel=\"nofollow noreferrer\">Jenkins</a> is a fork of <a href=\"http://hudson-ci.org/\" rel=\"nofollow noreferrer\">Hudson</a>.</p>\n\n<p>What are the benefits/advantages of using Jenkins over Hudson for a small private business?</p>\n",
	"OwnerUserId": 3,
	"LastEditorUserId": 210,

vanga / flask-server.py

Created September 8, 2022 15:01

Sample flask application with Elastic APM integration

	from flask import Flask
	from elasticapm.contrib.flask import ElasticAPM

	from flask import request
	app = Flask(__name__)
	app.config['ELASTIC_APM'] = {
	'SERVICE_NAME': 'flask',
	'SERVER_URL': 'https://<apm-server-ip/dns>.com:8200',
	'SERVER_CERT': 'path to elk.example.com/elk.example.com.crt',
	'SERVER_TIMEOUT': '5s',

vanga / apm-server.yml

Created September 8, 2022 14:56

APM server configuration with auth and SSL certs

	apm-server:
	host: "0.0.0.0:8200"
	auth:
	secret_token: 'secret token generated using openssl or any other string'
	ssl:
	enabled: true
	certificate: '/usr/share/apm-server/certs/elk.example.com/elk.example.com.crt'
	key: '/usr/share/apm-server/certs/elk.example.com/elk.example.com.key'

	output.elasticsearch:

vanga / kibana-ssl-config.yml

Created September 8, 2022 14:47

SSL configuration for Kibana

	server.ssl.enabled: true
	server.ssl.certificate: '/home/ubuntu/certs/elk.example.com/elk.example.com.crt'
	server.ssl.key: '/home/ubuntu/certs/elk.example.com/elk.example.com.key'

vanga / create-self-signed-certs.sh

Created September 8, 2022 14:41

Create self signed certificates using Elasticsearch util scripts

	wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.4.1-linux-x86_64.tar.gz
	tar -zxf elasticsearch-8.4.1-linux-x86_64.tar.gz
	cd elasticsearch-8.4.1/
	./bin/elasticsearch-certutil ca --pem
	unzip elastic-stack-ca.zip
	./bin/elasticsearch-certutil cert --ca-cert ca/ca.crt --ca-key ca/ca.key --pem --name elk.example.com
	unzip certificate-bundle.zip
	cd ../

Pradeep vanga