Skip to content

Instantly share code, notes, and snippets.

View mcanaves's full-sized avatar

Mateu Cànaves mcanaves

  • Belvo
  • Pollença
View GitHub Profile
@mcanaves
mcanaves / deregister_amis.py
Created September 14, 2018 21:11
Deregister old AMIs with boto3
"""
Deregister retired AMIs.
This script deregister all images marked as retired more than a X days
and removes the associated snapshot volumes.
To be able to identify the marked ones as retired and the time when it was
done the following tag system is used:
- stage [dev|production|retired]: used to track the life cycle of an image.
- state-changed [YYYYMMDDHHSS]: datetime mark to track when
stage changes were made.
@mcanaves
mcanaves / datadog-nginx-json
Last active July 11, 2017 15:48 — forked from gane5h/datadog-nginx
Nginx json log parsing with datadog
"""
Thanks to gane5h for the original script.
Custom parser for JSON nginx log suitable for use by Datadog 'dogstreams'.
To use, add to datadog.conf as follows:
dogstreams: [path to ngnix log (e.g: "/var/log/nginx/access.log"]:[path to this python script (e.g
"/usr/share/datadog/agent/dogstream/nginx.py")]:[name of parsing method of this file ("parse")]
so, an example line would be:
dogstreams: /var/log/nginx/access.log:/usr/share/datadog/agent/dogstream/nginx.py:parse
Log of nginx should be defined like that:
log_format access_json '{ "time": "$time_local", '
@mcanaves
mcanaves / missing_values_sequence.sql
Last active February 16, 2017 17:39
Find missing values in a sequence with SQL
SELECT start, stop FROM (
SELECT m.<number_field> + 1 AS start,
(SELECT min(<number_field>) - 1 FROM <table> AS x WHERE x.<number_field> > m.<number_field>) AS stop
FROM <table> AS m
LEFT OUTER JOIN <table> AS r ON m.<number_field> = r.<number_field> - 1
WHERE r.<number_field> IS NULL
) AS x
WHERE stop IS NOT NULL
ORDER BY start;
@mcanaves
mcanaves / duplicated_records.sql
Last active February 16, 2017 17:39
Search for duplicated records with SQL
SELECT <filter_column1>, ..., <filter_columnN>, count(*)
FROM <table>
GROUP BY <filter_column1>, ..., <filter_columnN>
HAVING count(*) > 1;
@mcanaves
mcanaves / query_schemas.sql
Last active January 25, 2017 14:53
Execute a query over all schemas on a postgres db
DO $$
DECLARE s record;
BEGIN
FOR s IN SELECT n.nspname AS schema
FROM pg_catalog.pg_namespace AS n
WHERE n.nspname NOT LIKE 'pg_%' AND n.nspname <> 'information_schema'
LOOP
RAISE NOTICE '%', s;
EXECUTE FORMAT('<QUERY ON %I>', s.schema);
END LOOP;
@mcanaves
mcanaves / reset.sql
Last active January 25, 2017 14:54 — forked from tbarbugli/reset.sql
Reset all sequences for all schemas on a postgres db
SELECT 'SELECT SETVAL(' ||quote_literal(S.relname)|| ', MAX(' ||quote_ident(C.attname)|| ') ) FROM ' ||quote_ident(T.relname)|| ';'
FROM pg_class AS S, pg_depend AS D, pg_class AS T, pg_attribute AS C
WHERE S.relkind = 'S'
AND S.oid = D.objid
AND D.refobjid = T.oid
AND D.refobjid = C.attrelid
AND D.refobjsubid = C.attnum
ORDER BY S.relname;