Last active
August 19, 2026 10:34
-
-
Save balzog/4942913ea4c12157479025fed942a495 to your computer and use it in GitHub Desktop.
memo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ### | |
| PERSONAL COLLECTION OF MEMO | |
| by https://github.com/niainaLens | |
| ### |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC --- | |
| # IT Roles | |
| + Architecture | |
| - Solution Architect | |
| design technical solution from business requirement | |
| - Infrastructure Architect | |
| IT foundation designer | |
| - Application Architect | |
| App infra designer | |
| + Build | |
| - DevOps Engineer: | |
| build automation/process/IaC/CICD | |
| - Platform Engineer | |
| DevOps++, create platform and tooling for Dev | |
| - SRE | |
| infrastructure stability holder | |
| # URL/Percent encoding | |
| -> https://en.wikipedia.org/wiki/Percent-encoding | |
| Reserved characters after percent-encoding | |
| ! # $ & ' ( ) * + , / : ; = ? @ [ ] | |
| %21 %23 %24 %26 %27 %28 %29 %2A %2B %2C %2F %3A %3B %3D %3F %40 %5B %5D | |
| Common characters after percent-encoding (ASCII or UTF-8 based) | |
| ␣ " % - . < > \ ^ _ ` { | } ~ £ € | |
| %20 %22 %25 %2D %2E %3C %3E %5C %5E %5F %60 %7B %7C %7D %7E %C2%A3 %E2%82%AC | |
| --- naming convention | |
| TYPE_SOMETHING_NUMBER | |
| ex: - a mailing list for group of mail | |
| ml_teamName@society.com | |
| - a bucket in google storage | |
| gs_PROJECT_LOCATION_NUMBER -> gs_project_ew1_1 | |
| - a server compute engine | |
| cmp_PROJECT_LOCATION_NUMBER -> cp_project_ew1_1 | |
| ## resources naming pattern | |
| -> https://stepan.wtf/cloud-naming-convention/ | |
| [prefix/org]-[resource_type]-[project]-[env]-[location]-[short_description]-[suffix/number] | |
| --- RELEASE MANAGEMENT --- | |
| # release strategy | |
| + Big Bang | |
| replace all with new at once - requiring downtime | |
| + Rolling | |
| updates gradually - no downtime, run both versions | |
| + Blue/Green | |
| uses two identical prod envs (live and idle) | |
| traffic switched to new env once verified | |
| + Canary | |
| releases new to few real users first to monitor for issues before rolling all out | |
| + Feature Toggles | |
| Deploys behind a configuration switch, | |
| allowing features to be turned on or off without deploying new code | |
| + Shadow | |
| duplicates live user traffic to the new version in the background to test performance | |
| then load without impacting the actual user experience. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## DevOps | |
| + LPIC DevOps Tools Engineer | |
| ## Cloud | |
| + GCP | |
| + AWS | |
| + AZURE | |
| ## Linux | |
| + LPIC 1, 2, 3 | |
| ## Network | |
| + CISCO |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| A | |
| *A/B Testing |Release management, Delivery, Marketing | |
| delivery of 2 versions of products | |
| *Active Directory |IAM, Microsoft | |
| Microsoft users directory service/database | |
| *Anthos |Cloud, GCP, Kubernetes | |
| GCP multicloud/unified kubernetes platform | |
| *App Engine |Cloud, Container, GCP | |
| GCP application runner, PaaS | |
| APM: Application Performance Monitoring |Monitoring, Elastic, AppDynamics, Dynatrace | |
| *AT-LEAST ONCE: |Message delivery, Queue, Message broke | |
| message will be delivered at least one time, but there may be duplicates | |
| *AT-MOST ONCE: |Message delivery, Queue, Message broker | |
| message is delivered at most one time, with the possibility of not being delivered at all | |
| *ATOMIC DESIGN: reusable action & workflow |Development, DevOps, Design | |
| B | |
| BAA: Business Associate Agreement |Cloud, Compliance, GCP | |
| BAFO: Best And Final Offer |Management, Project | |
| *Bastion |IAM, Server | |
| jump server/ gateway to manage access to private servers from external network | |
| BCP: Business Continuity Plan (*PCA) |Management, Agile | |
| C | |
| CAB: Change Advisory Board |Deployment | |
| *CC-BY-NC-SA: Licence Creative Commons |Licence | |
| *Canary Release |Release Management, Delivery | |
| delivery to restricted users before production | |
| CDN: Content Delivery Network |Dev, Network, Infrastructure | |
| *Circuit Breaker: |Resilience, Design pattern | |
| autostop failed service to prevent cascading failures in a distributed system | |
| CHANGELOG: List of all changes made to a project |Dev, Release Management | |
| *Cloud Identity |Cloud, IAM, GCP | |
| GCP IDasS to sync users from Active Directory | |
| *Cloud Run |Cloud, Container, GCP | |
| GCP managed container serverless | |
| CMEK: Customer-Managed Encryption Keys |Security, Cloud, GCP | |
| CPU: Central Processing Unit |Machine | |
| *CUTOVER PLAN: |Agile | |
| actions plan for a big change of old technology to a new one (go-live, go-prod) | |
| D | |
| DAN: Do Anything Now |AI | |
| *Dark Launch |Deployment, Release Management | |
| release feature to limited users/environments | |
| DoD: Definition Of Done |Agile, Scrum | |
| DBT: Data Build Tools |DBT, Data, Analytics | |
| DLQ: Dead Letter Queue |Queue | |
| *Dilbert Principle (*Peter Principle): |Management | |
| companies tend to promote incompetent employees to management to minimize their ability to harm productivity | |
| DRP: Disaster Recovery Plan |Cloud | |
| *DRY-RUN: |Testing, CI/CD | |
| run without applying | |
| *Dunbar Number: |Management, Agile | |
| theory that limit to number of people we can maintain stable social relationships at once | |
| DX: Developer eXperience |Development | |
| E | |
| *EXACTLY ONCE: |Message delivery, Queue, Message broker | |
| message is delivered exactly one time, without duplicates/omissions, then disappear | |
| *EXPONENTIAL BACKOFF: |Network, Algorithm | |
| progressively increases the time between successive retries | |
| F | |
| *FACILITY MANAGEMENT: |Management, Agile | |
| management supporting people, ensures the functionality, comfort, safety, sustainability and efficiency of the built environment | |
| FAIR: Findable, Accessible, Interoperable, Reusable |Data | |
| *FALLBACK: |Security, DevOps | |
| alternative option used when the primary plan is not successful/unavailable | |
| FTT: Failure To Tolerate |Security, Cloud | |
| G | |
| GCDS: Google Cloud Directory Sync |IAM, GCP | |
| *Gemba: |Management, Agile, Lean | |
| process to Go, Look and See, how the work is done | |
| GDPR: General Data Protection Regulation |Cloud, Compliance, Law | |
| GPO: Group POlicy |Security | |
| GPU: Graphical Processing Unit |Machine | |
| H | |
| HIPPA: Health Insurance Portability and Accountability Act |Cloud, Compliance | |
| HPC: High Performance Computing | |
| *HUSHLOGIN: message on login banner in Linux for individual user (suppress MOTD) |Linux | |
| I | |
| IAP: Identity-Aware Proxy |IAM, Cloud, GCP | |
| *Ikigai |Personal Development | |
| Japanese concept for perso dev | |
| *Iron Triangle: |Project management | |
| Project management triangle with triple constraints: Cost, Scope, Price | |
| *ISO 26000: | |
| Norm RSE |RSE | |
| *ISO 27001: |Security | |
| international standard for information systems security to define the Information Security Management System (ISMS) | |
| ISMS: Information Security Management System |Security | |
| I&A: Inspect & Adapt |Agile, SAFe | |
| J | |
| K | |
| KB: Knowledge Base |Agile, Doc | |
| KISS: Keep It Simple Stupid |Agile | |
| KMS: Key Management Service |Network, Security | |
| L | |
| *LEAST PRIVILEGE: |IAM, GCP, Security | |
| assign role with only strict necessary perms at the smallest scope/resource | |
| *LIFT AND SHIFT (*REHOSTING/ REPLATFORMING): |Cloud, Migration | |
| Move server to cloud as it is, without redesign | |
| *LIVENESS PROBE |Kubernetes, Probe, Healthcheck | |
| determine if the application is running and responsive (not accept traffic yet, see Readiness Probe) | |
| LUKS: Linux Unified Key Setup |Linux | |
| M | |
| MEAT: Most Economically Advantageous Tender |Management, Project | |
| MFA: Multi-Factor Authentication |Cloud, IAM, Security | |
| MIG: Managed Instances Group |Cloud, Compute, GCP | |
| MTBF: Mean Time Before Failure |Infrastructure, Metric | |
| MTTA: Mean Time To Acknowledge |Infrastructure, Metric | |
| MTTF: Mean Time To Failure |Infrastructure, Metric | |
| MTTR: Mean Time To Recovery |Infrastructure, Metric | |
| MVP: Minimum Viable Product |Development, Agile | |
| N | |
| O | |
| ORM: Object-Relational Mapping |Database | |
| *OCKHAM'S RAZOR: |Agile, Principle, DevOps | |
| less is better, eliminate all unlikely hypothesis, parsimony principle | |
| -> https://www.linkedin.com/posts/jeanbaptistemusso_rasoir-dockham-wikip%C3%A9dia-activity-7019270010028052480-txnE?utm_source=share&utm_medium=member_desktop | |
| P | |
| *Parquet |Apache, BigData, Format | |
| Apache BigData file format, column-oriented data file | |
| *PETER PRINCIPLE: (*Dilbert Principle) |Management | |
| employees are promoted based on success until they attain their "level of incompetence" and are no longer successful | |
| PCA: Plan de Continuite d'Activite (*BCP) |Management, Agile | |
| PCI-DSS: Payment Card Industry Data Security Standard |Security, RGPD | |
| PDCA: Plan Do Check Adjust |Agile | |
| PoLP: PRINCIPLE of LESS PRIVILEGE |Security | |
| PoP: Point Of Presence |CDN, Cloud | |
| *PORT-KNOCKING: |Network, Security, Firewall | |
| open closed ports by sending network packets containing special information | |
| PQQ: Pre-Qualification Questionnaire |Management, Project | |
| *Preemptible/Spot VM |Cloud, Compute, GCP | |
| GCP discounted/stateless/batch/fault-tolerant VM | |
| Q | |
| *QUORUM: |Management | |
| minimum number of people needed to hold meetings or make decisions during certain company meetings | |
| R | |
| RACI: Responsible, Accountable, Consulted, Informed |Management | |
| RaMP: Rapid Migration and Modernization Program |Cloud, GCP, Migration | |
| RAT: Riskiest Assumption Test |Development, Agile, Test | |
| RBAC: Role-Based Access Control |IAM, Role, Permission, Cloud | |
| RCU: Referentiel Client Unique (*SCV) |Database, Marketing | |
| *READINESS PROBE |Kubernetes, Probe, Healthcheck | |
| determine if a container is ready to accept traffic and to be plugged to the system (after Liveness Probe OK) | |
| *REHOSTING (*LIFT AND SHIFT/ REPLATFORMING): |Cloud, Migration | |
| Move server to cloud as it is, without redesign | |
| *RELEASE NOTE |Dev, Release Management | |
| Document with new or upgraded product | |
| REPL: Read-Eval-Print Loop |Programming | |
| *REPLATFORMING (*LIFT AND SHIFT/ REHOSTING): |Cloud, Migration | |
| Move server to cloud as it is, without redesign | |
| *RESILIENCE: |Security | |
| adaptation to crash, I crash Others not | |
| RETEX: Retour d'Experience |Management, Agile | |
| RFI: Request For Information |Management, Project | |
| RFP: Request For Proposal |Management, Project | |
| RFQ: Request For Quotation |Management, Project | |
| ROAM: Resolved Owned Accepted Mitigated |Management, Agile, SAFe, Risk management | |
| RPO: Recovery Point Objective |Management, Agile | |
| RTE: Release Train Engineer |Management, Agile, SAFe | |
| RTO: Recovery Time Objective |Management, Agile | |
| S | |
| SAST: Static Application Security Testing |Security | |
| SCIM: System for Cross-domain Identity Management |IAM, Identity, Cloud | |
| SCV: Single Client View (*RCU) |Database, Marketing | |
| *Service Account |Cloud, IAM, GCP | |
| non-human account for apps (authent, secure interact) | |
| SKU: Stock Keeping Units |Azure, Cloud, Microsoft | |
| SLA: Service Level Agreement |Agile, Project management, Support service | |
| SLC: Simple Lovable Complete |Agile, Project management | |
| *SLASHDOT EFFECT/ SLASDOTTING |Security | |
| temporary surge in traffic to a website | |
| SLO: Service Level Objectives |Agile, Project management | |
| SNS: Simple Notification Service |AWS | |
| SSS: Shamir's Secret Sharing |Vault, Security, Encryption | |
| *Stratozone |Cloud, Migration | |
| migration assessment platform | |
| *Strangler fig pattern |Architecture, Cloud, Migration | |
| architecture to migrate monolythic app to microservices | |
| T | |
| *Tensor |Machine Learning | |
| Multi-dimensional matrix | |
| TDD: Test Driven Development |Agile, Principle, Dev | |
| TPU: Tensor Processing Unit |Machine, Machine Learning | |
| Two-SV: Two-Steps Verification (*2-SV) |Security, Cloud | |
| U | |
| DX: User eXperience |Development | |
| V | |
| VPC: Virtual Private Cloud |Cloud, Network | |
| VPC-SC: VPC Service Control |Cloud, GCP, Network | |
| *VPC peer |Cloud, Network, GCP | |
| interconnect 2 VPCs | |
| *VPC shares |Cloud, Network, GCP | |
| VPS shared by multiple projects | |
| W | |
| *Wideband Delphi |Management | |
| Estimation method, diverge-converge | |
| WAF: Web Application Firewall |Infrastructure | |
| *Watermelon Metrics |Agile, Marketing | |
| vanity metrics, hype but not representative metrics | |
| WSJF: Weighted Shortest Job First |Agile, SAFe | |
| X | |
| Y | |
| YAGNI: You Ain't Gonna Need It |Agile, Principle, DevOps | |
| Z | |
| *Zero Trust |Security | |
| strict access controls and not trusting anyone by default, continuous validation on every stage of interaction | |
| 0 | |
| 1 | |
| 2 | |
| 2-SV: 2-Steps Verification (Two-SV) |Security, Cloud | |
| 3 | |
| 3W: Who can do What in Which resource |Cloud, GCP, IAM, Security | |
| 4 | |
| 5 | |
| 6 | |
| 7 | |
| 8 | |
| 9 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- DOCKER | |
| ## docker in wsl/wsl2 | |
| "failed to start daemon: Error initializing network controller ... iptable" | |
| --> https://github.com/microsoft/WSL/issues/6655 | |
| $ sudo update-alternatives --set iptables /usr/sbin/iptables-legacy | |
| $ sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy | |
| $ sudo dockerd & | |
| or | |
| $ sudo dockerd --debug --iptables=false & |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --> https://github.com/awesome-selfhosted/awesome-selfhosted | |
| --> https://github.com/awesome-selfhosted/awesome-selfhosted/blob/master/non-free.md | |
| ==================== | |
| == MIDDLEWARE == | |
| ==================== | |
| --- ARCHITECTURE | |
| - OPEX | |
| --- INFRA AS CODE | |
| ## Configuration management | |
| - ANSIBLE | |
| - chef | |
| - puppet | |
| ## Infrastructure build | |
| - TERRAFORM | |
| - Cloud Formation | |
| ## Image build | |
| - PACKER | |
| - DOCKER | |
| ## Containerization | |
| - DOCKER | |
| - Cri-o | |
| ## Container orchestrator | |
| - DOCKER-COMPOSE | |
| - DOCKER-SWARM | |
| + KUBERNETES | |
| - operator: | |
| * kube-green (auto shutdown) | |
| - validator/lint/syntax-check | |
| * kubeconform | |
| * kubescore | |
| --- DEV | |
| ## SCM | |
| - Git | |
| - Mercurial | |
| ## SCM GUI | |
| - BitBucket |CI/CD | |
| - Github |CI/CD | |
| - GITLAB |CI/CD | |
| ## Test code quality review | |
| - SONAR/SONARQUBE | |
| - JUNIT | |
| - Selenium | |
| --- CODE COLLABORATION/REVIEW | |
| - GITLAB | |
| - Gerrit | |
| - Gitea | |
| --- CI/CD | |
| - GITLAB-CI | |
| - Jenkins | |
| - Rundeck | |
| --- CMDB | |
| - Git (YAML) | |
| - Insight (JIRA) | |
| --- SECURITY | |
| ## Vault | |
| - ansible-vault | |
| - VAULT (hashicorp) | |
| ## WAF | |
| - Wordfence (wordpress plugin) | |
| ## FRAUD | |
| - DataDome (fraud protection) | |
| --- DATABASE | |
| ## RDBMS | |
| - MySQL | |
| - PostgreSQL | |
| - OracleDB | |
| - CLOUDSQL | |
| ## NoSQL | |
| - MongoDB | |
| ## Database config management | |
| - Liquibase | |
| - SonarQube | |
| ## Time series | |
| - influxDB | |
| --- SECURITY | |
| ## Static application security testing (SAST) | |
| - Gitleaks | |
| --- TEST | |
| ## End-to-end test | |
| - Playwright | |
| - MrSuricate | |
| - Selenium | |
| ## load/performance testing | |
| - Apache JMeter | |
| - NeoLoad | |
| --- IDE/TOOLS | |
| ## FOR DOCKER | |
| - Portainer | |
| ## FOR KUBERNETES | |
| - Lens | |
| ## IDE | |
| - VSCode | |
| --- IDENTITY MANAGEMENT | |
| - LDAP | |
| - Keycloak | |
| - Auth0 | |
| - Authentik | |
| --- PROJECT MANAGEMENT | |
| ## ITIL | |
| - Redmine | |
| ## AGILE | |
| - JIRA |Ticketing | |
| --- TICKETING/ HELPDESK | |
| - GLPI | |
| - JIRA | |
| --- MONITORING | |
| ## Cloud provider | |
| - GCP: Stackdriver | |
| - AWS: | |
| - Azure: | |
| ## Container monitor | |
| - cadvisor | |
| - telegraf | |
| ## Data | |
| - Datadog | |
| - Splunk | |
| ## Observability | |
| - Kibana | |
| - Grafana | |
| - Dynatrace | |
| --- INVENTORY | |
| - GLPI |Ticketing | |
| - OCS Inventory | |
| --- DOCUMENTATION | |
| - Wiki | |
| - DokuWiki | |
| - Wiki.js | |
| - Alfresco | |
| - ReadTheDocs | |
| /SaaS/ | |
| - Confluence (JIRA) | |
| - SharePoint | |
| --- COMMUNICATION | |
| - Slack | |
| - Discord | |
| - Teams | |
| - Postfix | |
| - Zimbra |Zimbra Cloud | |
| --- OPEN/PROPRIETARY | |
| ## Container | |
| Kubernetes --> Borg |GKE, EKS, AKS | |
| ## Database | |
| HBase |GCP BigTable | |
| ==================== | |
| == SAAS == | |
| ==================== | |
| --- AGILE | |
| - Klaxoon | |
| - Gather town | |
| --- Code collaboration | |
| - GITLAB | |
| - Github | |
| - BitBucket | |
| --- CI/CD | |
| - GITLAB-CI | |
| - Github Action | |
| - Azure DevOps | |
| --- CMDB | |
| - Gitlab/Github | |
| - Insight (JIRA) | |
| --- IDENTITY MANAGEMENT | |
| ## Single-Sign-On | |
| - Auth0 | |
| - Office365 | |
| --- REGISTRY | |
| ## Container/Image registry | |
| - Amazon ECR (Elastic Container Registry) |AWS | |
| - Azure Container Registry |Azure | |
| - Dockerhub | |
| - Google Artifact Registry |GCP | |
| - ttl.sh | |
| --- SHIPPING | |
| - UPS | |
| - Chronopost | |
| - Mondial Relay | |
| --- PAYMENT | |
| - Paypal | |
| - Payline | |
| - Adyen | |
| --- TAX | |
| - Avalara | |
| --- TICKETING | |
| ## ITIL | |
| - ServiceNow | |
| ## AGILE | |
| - JIRA | |
| - Trello | |
| --- DNS REGISTRAR | |
| - Akamai | |
| - Gandi | |
| - GoDaddy | |
| - LWS | |
| ## Mail service | |
| - MS Office365 | |
| - Google Workspace | |
| - Zimbra Cloud | |
| ## Mail marketing | |
| - Google Workspace | |
| - SFMC (SalesForce Marketing Cloud) | |
| - Mailjet | |
| - Mailchimp | |
| - Sendgrid | |
| --- DOCUMENTATION | |
| - Confluence (JIRA) | |
| - SharePoint | |
| - ReadTheDocs | |
| --- MISC | |
| - Bazzarvoice | |
| rating & opinion to e-commerce product |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- SYSTEM VARS | |
| ARCH=$(dpkg --print-architecture) | |
| KERNEL_NAME=$(uname -s) | |
| KERNEL_RELEASE=$(uname -r) | |
| LSB_RELEASE_OS=$(lsb_release -is) | |
| LSB_RELEASE_VERSION=$(lsb_release -rs) | |
| LSB_RELEASE_CODENAME=$(grep -oP '(?<=UBUNTU_CODENAME=).*' /etc/os-release || lsb_release -cs) | |
| DIR_KEYRING="/etc/apt/keyrings" | |
| DIR_GPG="/etc/apt/trusted.gpg.d" | |
| --- APP | |
| ## dependencies | |
| + apt package | |
| - apt-transport-https |code, gcloud, helm, kubectl, sublime-text | |
| - bash-completion |bash, kubectl | |
| - build-essential | |
| - ca-certificates |docker, gcloud, kubetcl | |
| - curl |docker, kubectl, teams | |
| - file | |
| - git | |
| - gpg |code | |
| - gnupg |gcloud | |
| - gnome-keyring |VSCode* (https://code.visualstudio.com/docs/editor/settings-sync#_troubleshooting-keychain-issues) | |
| - gnupg |docker | |
| - lsb-release |docker | |
| - procps | |
| - python3 |ansible | |
| - python3-pip |ansible | |
| - vim | |
| - wget |code | |
| + pip package | |
| - openai>=0.18.1 |codex-cli | |
| - psutil>=5.9.0 |codex-cli | |
| ## package manager | |
| + apt key (root:root 644) | |
| - ansible | |
| $ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 93C4A3FD7BB9C367 | |
| - brave | |
| $ curl -fsSLo ${DIR_KEYRING}/brave-browser.gpg https://brave-browser-apt-release.s3.brave.com/brave-browser-archive-keyring.gpg | |
| - docker | |
| $ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o ${DIR_KEYRING}/docker.gpg | |
| $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o ${DIR_KEYRING}/docker.asc | |
| - hashicorp (terraform, vault, consul, packer) | |
| $ wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee ${DIR_KEYRING}/hashicorp-archive-keyring.gpg | |
| - helm | |
| $ curl https://baltocdn.com/helm/signing.asc | sudo gpg --dearmor -o ${DIR_KEYRING}/helm.gpg | |
| - gcloud | |
| $ | |
| - google-cloud (gcloud, kubernetes) | |
| $ curl -fsSLo ${DIR_KEYRING}/google-cloud.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg | |
| - microsoft (code, teams) | |
| $ curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | sudo gpg --dearmor -o ${DIR_KEYRING}/packages.microsoft.gpg | |
| - sublimetext | |
| $ wget -qO - https://download.sublimetext.com/sublimehq-pub.gpg | sudo gpg --dearmor -o ${DIR_GPG}/sublimehq-archive.gpg | |
| + sources.list.d (*.list) | |
| - ansible | |
| deb http://ppa.launchpad.net/ansible/ansible/ubuntu focal main | |
| - brave-browser | |
| deb [signed-by=${DIR_KEYRING}/brave-browser.gpg] https://brave-browser-apt-release.s3.brave.com/ stable main | |
| - chrome | |
| deb [arch=${ARCH}] https://dl.google.com/linux/chrome/deb/ stable main | |
| - code | |
| deb [arch=${ARCH} signed-by=${DIR_KEYRING}/microsoft.gpg] http://packages.microsoft.com/repos/code stable main | |
| - docker | |
| deb [arch=${ARCH} signed-by=${DIR_KEYRING}/docker.gpg] https://download.docker.com/linux/debian ${LSB_RELEASE_CODENAME} stable | |
| - hashicorp (terraform, packer, vault, consul) | |
| deb [signed-by=${DIR_KEYRING}/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com ${LSB_RELEASE_CODENAME} main | |
| - helm | |
| deb [arch=${ARCH} signed-by=${DIR_KEYRING}/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main | |
| - gcloud | |
| deb [signed-by=${DIR_KEYRING}/google-cloud.gpg] https://packages.cloud.google.com/apt cloud-sdk main | |
| - kubernetes | |
| deb [signed-by=${DIR_KEYRING}/kubernetes.gpg] https://apt.kubernetes.io/ kubernetes-xenial main | |
| - sublimetext | |
| deb https://download.sublimetext.com/ apt/stable/ | |
| - teams | |
| deb [signed-by=${DIR_KEYRING}/microsoft.gpg] https://packages.microsoft.com/repos/ms-teams stable main | |
| + package | |
| - ansible | |
| - brave-browser | |
| - code | |
| - docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin | |
| - docker-compose | |
| - google-chrome-stable | |
| - helm | |
| - jq | |
| - kubectl | |
| - shellcheck | |
| - snap | |
| - sublimetext | |
| - teams | |
| - terraform | |
| ## deb | |
| - wps | |
| ## snap | |
| - skype | |
| ## git repo | |
| - brew | |
| $ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" | |
| - codex-cli | |
| https://github.com/microsoft/Codex-CLI/blob/main/Installation.md | |
| - yq | |
| $ wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq &&\ | |
| chmod +x /usr/bin/yq | |
| --- CONFIG |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- ACER ASPIRE 5 A514-55-51XE | |
| OS: Debian 11 | |
| Default kernel: 5.10 | |
| DE: KDE Plasma | |
| ## ISSUES | |
| + with KDE Plasme DE, black screen on startup login screen, no DE | |
| + log | |
| $ dmesg --level=err | |
| [ 0.337937] pci 0000:00:07.0: DPC: RP PIO log size 0 is invalid | |
| [ 1.544938] integrity: Problem loading X.509 certificate -65 | |
| [ 1.544948] integrity: Problem loading X.509 certificate -65 | |
| [ 2.267560] i915 0000:00:02.0: firmware: failed to load i915/adlp_dmc_ver2_16.bin (-2) | |
| [ 2.267563] firmware_class: See https://wiki.debian.org/Firmware for information about missing firmware | |
| [ 2.267567] i915 0000:00:02.0: firmware: failed to load i915/adlp_dmc_ver2_16.bin (-2) | |
| [ 2.277183] i915 0000:00:02.0: firmware: failed to load i915/adlp_guc_70.1.1.bin (-2) | |
| [ 2.277189] i915 0000:00:02.0: firmware: failed to load i915/adlp_guc_70.1.1.bin (-2) | |
| [ 2.277194] i915 0000:00:02.0: firmware: failed to load i915/adlp_guc_69.0.3.bin (-2) | |
| [ 2.277199] i915 0000:00:02.0: firmware: failed to load i915/adlp_guc_69.0.3.bin (-2) | |
| [ 2.277200] i915 0000:00:02.0: GuC firmware i915/adlp_guc_70.1.1.bin: fetch failed with error -2 | |
| [ 2.368614] i915 0000:00:02.0: GuC initialization failed -2 | |
| [ 2.368616] i915 0000:00:02.0: Enabling uc failed (-5) | |
| [ 2.368617] i915 0000:00:02.0: Failed to initialize GPU, declaring it wedged! | |
| [ 6.918799] uvcvideo 1-7:1.1: Failed to query (129) UVC probe control : 26 (exp. 48). | |
| [ 6.918833] uvcvideo 1-7:1.1: Failed to initialize the device (-5). | |
| [ 7.257953] bluetooth hci0: firmware: failed to load rtl_bt/rtl8852bu_fw.bin (-2) | |
| [ 7.257990] bluetooth hci0: firmware: failed to load rtl_bt/rtl8852bu_fw.bin (-2) | |
| [ 7.258014] Bluetooth: hci0: RTL: firmware file rtl_bt/rtl8852bu_fw.bin not found | |
| [ 7.507427] sof-audio-pci-intel-tgl 0000:00:1f.3: firmware: failed to load intel/sof/sof-adl.ri (-2) | |
| [ 7.507471] sof-audio-pci-intel-tgl 0000:00:1f.3: firmware: failed to load intel/sof/sof-adl.ri (-2) | |
| [ 7.507506] sof-audio-pci-intel-tgl 0000:00:1f.3: error: sof firmware file is missing, you might need to | |
| [ 7.507538] sof-audio-pci-intel-tgl 0000:00:1f.3: download it from https://github.com/thesofproject/sof-bin/ | |
| [ 7.507557] sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to load DSP firmware -2 | |
| [ 7.507965] sof-audio-pci-intel-tgl 0000:00:1f.3: error: sof_probe_work failed err: -2 | |
| [ 11.372617] r8169 0000:2b:00.0: firmware: failed to load rtl_nic/rtl8168h-2.fw (-2) | |
| [ 11.373357] r8169 0000:2b:00.0: firmware: failed to load rtl_nic/rtl8168h-2.fw (-2) | |
| [ 11.373912] r8169 0000:2b:00.0: Unable to load firmware rtl_nic/rtl8168h-2.fw (-2) | |
| ## KERNEL UPGRADE | |
| + add backport repo to sources.list | |
| deb http://deb.debian.org/debian bullseye-backports main contrib non-free | |
| deb-src http://deb.debian.org/debian bullseye-backports main contrib non-free | |
| + upgrade kernel | |
| $ apt update | |
| $ apt-cache policy linux-image-amd64 | |
| $ apt install linux-image-amd64=6.0.12-1~bpo11+1 linux-headers-6.0.0-0.deb11.6-amd64 | |
| $ reboot | |
| ## INSTALL DEVICES DRIVERS | |
| REQUIREMENTS | |
| // install firmwares from Backports | |
| $ apt install -t bullseye-backports firmware-linux firmware-linux-free firmware-linux-nonfree firmware-sof-signed firmware-intel-sound firmware-misc-nonfree firmware-realtek | |
| // download latest firmware from kernel git repo | |
| $ git clone https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git /opt/linux-firmware | |
| (Backup existing firmware files if needed) | |
| VGA (Intel Iris Xe Graphics) | |
| --> Intel Corporation Device 46a8 (rev 0c) | |
| --> i915 | |
| $ mv /lib/firmware/i915 /lib/firmware/i915_ORIG | |
| $ cp -avr /opt/linux-firmware/i915 /lib/firmware/ | |
| BLUETOOTH (Realtek) | |
| --> Realtek Bluetooth Radio | |
| --> rtl_bt | |
| $ mv /lib/firmware/rtl_bt /lib/firmware/rtl_bt__ORIG | |
| $ cp -avr /opt/linux-firmware/rtl_bt /lib/firmware/ | |
| AUDIO (Intel) | |
| --> Intel Corporation Device 51c8 (rev 01) | |
| --> SOF Project | |
| // Download the SOF firmware project binaries from repo | |
| $ wget https://github.com/thesofproject/sof-bin/releases/download/v2.2.4/sof-bin-v2.2.4.tar.gz -O - |tar -xz -C /opt/ | |
| $ mv /lib/firmware/intel/sof /lib/firmware/intel/sof_ORIG | |
| $ mv /lib/firmware/intel/sof-tplg /lib/firmware/intel/sof-tplg_ORIG | |
| $ mv /lib/firmware/intel/sof-tplg-v1.7 /lib/firmware/intel/sof-tplg-v1.7_ORIG | |
| $ cd /opt/sof-bin-v2.2.4 && ./install.sh v2.2.4 | |
| ETHERNET (Realtek) | |
| --> Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15) | |
| --> r8168 | |
| ... NO ISSUE ... | |
| WIFI (Realtek) | |
| --> Realtek Semiconductor Co., Ltd. Device b852Realtek Bluetooth Radio | |
| --> rtw89 | |
| // install requirements | |
| $ apt install make gcc linux-headers-$(uname -r) build-essential git | |
| // Download Realtek rtlwifi codes from project repo | |
| - kernel <6.0 : https://github.com/HRex39/rtl8852be | |
| - kernel >=6.0 : https://github.com/lwfinger/rtw89 | |
| $ git clone https://github.com/lwfinger/rtw89.git /opt/rtw89 | |
| $ cd /opt/rtw89 | |
| $ make | |
| $ make install | |
| // copy new firmwares files from kernel repo | |
| $ mv /lib/firmware/rtw89 /lib/firmware/rtw89_ORIG | |
| $ cp -avr linux-firmware/rtw89 /lib/firmware/rtw89 | |
| // unload/reload kernel modules | |
| $ modprobe -rv rtw_8852ae | |
| $ modprobe -rv rtw_core | |
| $ modprobe -v rtw_8852ae | |
| THUNDERBOLT USB (Intel) | |
| --> Intel Corporation Device 463e (rev 04) (thunderbolt) | |
| --> thunderbolt | |
| ... NO ISSUE .. | |
| WEBCAM (ACER) | |
| --> ACER HD User Facing | |
| ### | |
| ### BIG PROBLEM | |
| ### | |
| https://www.mail-archive.com/kernel-packages@lists.launchpad.net/msg494663.html | |
| // reload kernel config | |
| $ update-initramfs -u -k all | |
| $ reboot | |
| --- ENV INSTALL --- | |
| 1- visudo | |
| 2- ssh keys | |
| + ssh-keygen | |
| + send key.pub @github | |
| 3- bashrc | |
| + git clone git@github.com:niainaLens/my-custom-env-rc.git | |
| + .bashrc | |
| ## Load custom env config | |
| source ~/my-custom-env-rc/perso.rc | |
| 4- python | |
| + install latest version | |
| + set update-alternatives |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## ELK | |
| + apm + fleetserver |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## PAYMENT | |
| - Stripe | |
| - Adyen | |
| ## SHIPPING | |
| - Chronopost | |
| - UPS | |
| ## TAX | |
| - Avalara | |
| ## MONITORING | |
| + Data | |
| - Datadog | |
| + APM | |
| - AppDynamics |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| + eBPF, Cilium | |
| + observability | |
| --- DATA | |
| + DBT (Data Build Tools) | |
| --- TERRAFORM | |
| + workspace | |
| --- ANSIBLE | |
| + collection | |
| --- GCP | |
| + service mesh | |
| + GKE Dataplane V2 | |
| --- PROMETHEUS | |
| + node exporter |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| I&A: Inspect & Adapt | |
| MMP: Minimum Marketable Product | |
| MVP: Minimum Viable Product | |
| PDCA: Plan Do Check Adjust | |
| ROAM: Resolved Owned Accepted Mitigated | |
| RPO: Recovery Point Objective | |
| RTE: Release Train Engineer | |
| RTO: Recovery Time Objective | |
| WSJF: Weighted Shortest Job First | |
| # retro | |
| KISS: Keep Improve Start Stop | |
| # Story/stress Points | |
| -> abstract estimation | |
| + take in account | |
| - difficulty | |
| - effort | |
| - risks | |
| - doubts/incertitudes | |
| - inter-dependancies | |
| + how to use | |
| - use fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233 | |
| >8 -> to cut | |
| - use t-shirt size: XS, S, M, L, XL, XXL | |
| >XL -> to cut | |
| - days/man: 1/2d, 1d, 2d, 3d, 4d, 5d, 6d | |
| >5d -> to cut | |
| # Planning Poker | |
| @ product backlog refinement - grooming | |
| US size estimation from relative effort | |
| - US presentation: features, requirements, by PO | |
| - clarification: team ask questions for hidden complexity, risks | |
| - private vote: with cards (Fibonacci sequence: 1, 2, 3, 5, 8, 13, t-shirt size) = relative effort | |
| - simultaneous Reveal | |
| - discussion: highest and lowest voters explain their reasoning | |
| - re-vote: until consensus is reached | |
| # Grooming/refinement --> Product Backlog Refinement | |
| -> review of team's backlog (backlog refinement) | |
| # Extreme Quotation | |
| # Burndown chart | |
| # Cut-Over planning | |
| Actions planning for a big change of an old technology to a new one | |
| --- MEETING | |
| # DSM (<=15m) | |
| - what you've done, what you are doing/ will do, if there is an issue? | |
| - turn by turn | |
| - no debate, no problem resolution | |
| - can tell if there is issues/problems -> debate after | |
| (2h Ceremony) | |
| # REVIEW | |
| + Sprint (<=1h20m) | |
| # PLANNING | |
| + PI | |
| + Sprint (<=30m) | |
| # RETRO | |
| + Sprint (<=10m) | |
| --- SPRINT | |
| # sprint planning | |
| 1- Backlog grooming | |
| 2- Sprint planning | |
| 3- Sprint | |
| 4- Sprint review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| AI Assist *(search) | |
| -> Vector DB + RAG *(context) | |
| -> ML + LLM *(action) | |
| -> AI Agent *(deploy) | |
| # Machine Learning (ML) | |
| systems learn to recognize patterns and make decisions from data | |
| EX: Music app | |
| -> lean listening habbit -> send recommencdation | |
| # Generative AI | |
| create new content from ML (based on learned patterns) -> text, code, image, video, audio | |
| EX: Create image from prompt | |
| # LLM (Large Language Models) | |
| Massive AI models trained on vast amounts of text | |
| Generative AI designed to understand/predict/generate human language | |
| EX: Summarize doc | |
| # AI Agents | |
| autonomous system using LLM to analyse environment, reason to make decision | |
| EX: Personnal assistant | |
| -> read mail -> summarize + make appointment | |
| # AI Assistant | |
| AI agent designed for human interaction, answer questions | |
| EX: Copilot, Gemini, ChatGPT, ClaudeCode | |
| # RAG (Retrieval-Augmented Generation) | |
| framework to improve LLM result by retrieving true information from external private database | |
| Give factual answer | |
| EX: employee search for his enterprise Security Policies | |
| -> summarize | |
| # Vectorization | |
| convert data (text, image, audio) to math vectors to help system for semantinc understanding | |
| EX: Semantic search in e-commerce | |
| -> "crimson jogging sneakers" -> "red running shoes" | |
| # Model Evaluation | |
| test AI model's performance using specific metrics (accuracy, coherence, fluency, safety) | |
| EX: evaluate AI precision | |
| -> test already-know products types | |
| # Governance | |
| frameworks, policies, ethical guidelines put in place to manage AI systems | |
| EX: AI can recommend credit approvals | |
| -> but can't auto deny without human audit | |
| --- USE CASE | |
| # AI cloud infrastructure management | |
| 0- Gouvernance | |
| define org policies and standards | |
| @ critical info can't be exposed | |
| -> drop WARNING if occurs | |
| *1- AI Assistant | |
| use code assistant for IaC | |
| @ clean/optimized code | |
| *1- Generative AI/LLM | |
| describe wanted svc/infra in natural language | |
| @ "Create GCP Project, with a VPC, and a securized VM" | |
| -> LLM generate Terraform Code | |
| 2- Vectorization | |
| ajust natural language prompt to technical meaning | |
| @ "securized VM" | |
| -> VM without public IP, access only with private cert | |
| 3- RAG | |
| the model ajust with org security policies | |
| @ auto added org network policies, tags, naming convention | |
| -> generate NEW adapted code | |
| *4- ML | |
| learn the common usage pattern of all cloud ressource | |
| @ generate data | |
| *4- AI Agent | |
| auto monitor cloud resource utilization from ML for cost-saving measure | |
| @ test & adjust instance size | |
| -> suggest new size & generate PR validation | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- DOC | |
| # architecture | |
| + agentless | |
| master node -- ssh --> node 1 | |
| ---------> node N | |
| # code directory structure | |
| ansible-project/ | |
| ansible.cfg | |
| requirements.yml | |
| main.yml | |
| hosts.ini | |
| inventories/ | |
| dev/ | |
| hosts.yml | |
| group_vars/ | |
| host_vars/ | |
| prod/ | |
| ... | |
| playbooks/ | |
| site.yml | |
| deploy.yml | |
| provision.yml | |
| roles/ | |
| base/ | |
| nginx/ | |
| app/ | |
| vault/ | |
| dev_vault_pass | |
| prod_vault_pass | |
| scripts/ | |
| pre-deploy.sh | |
| post-deploy.sh | |
| README.md | |
| # coebase structure (.yml) | |
| --- | |
| - name: Ensure Nginx is installed and at the latest version <-- task name | |
| ansible.builtin.package: <-- module | |
| name: nginx <-- module parameter | |
| state: latest | |
| become: true <-- task directive | |
| when: ansible_os_family == "Debian" | |
| notify: Restart Nginx | |
| tags: | |
| - webserver | |
| - setup | |
| ... | |
| # scaling Ansible | |
| when managing multiples nodes: | |
| - serial execution: prevent overloading servers | |
| - async tasks: for long-running operations | |
| - delegation: for central tasks, avoid repeated heavy tasks across all nodes | |
| - split roles/playbooks: small/reusable components | |
| - monitor performance (Ansible Tower / AWX orchestration) | |
| --- MISC --- | |
| ## suppress output of a task | |
| - task | |
| no_log: True | |
| --- FACTS | |
| # get all facts | |
| command | |
| $ ansible -m setup | |
| playbook | |
| - name: task | |
| gather_facts: yes | |
| # get specific facts | |
| - name: specific gather facts | |
| ansible.builtin.setup: | |
| gather_subset: | |
| - hardware | |
| - network | |
| # Custom Local Facts (namespace -> ansible_local) | |
| -> /etc/ansible/facts.d/env.fact | |
| [app] | |
| environment=production | |
| version=1.4.2 | |
| -> access env var via: ansible_facts['ansible_local']['preferences']['app']['environment'] | |
| # runtime facts -> set_fact Module | |
| - name: Determine dynamic path | |
| ansible.builtin.set_fact: | |
| custom_install_path: "/opt/app/{{ ansible_facts['distribution'] | lower }}" | |
| --- HANDLERS | |
| services triggered by changes | |
| # with playbook/task | |
| tasks: | |
| - name: Copy Nginx configuration file | |
| ansible.builtin.copy: | |
| src: ./files/nginx.conf | |
| dest: /etc/nginx/nginx.conf | |
| owner: root | |
| group: root | |
| mode: '0644' | |
| notify: Restart Nginx # 1. This tells Ansible to trigger the handler if this task changes | |
| handlers: | |
| - name: Restart Nginx # 2. This name MUST exactly match the 'notify' value above | |
| ansible.builtin.service: | |
| name: nginx | |
| state: restarted | |
| # with roles | |
| roles/ | |
| nginx_config/ | |
| files/ | |
| nginx.conf <-- Your actual Nginx config file goes here | |
| handlers/ | |
| main.yml <-- Your handler goes here | |
| tasks/ | |
| main.yml <-- Your task goes here | |
| --- INVENTORY --- | |
| # set inventory | |
| $ ansible-playbook example.yml -i /path/to/INVENTORY.file | |
| ENV VAR | |
| $ export ANSIBLE_INVENTORY=/path/to/INVENTORY.file | |
| ANSIBLE.CFG | |
| [defaults] | |
| inventory = /path/to/INVENTORY.file | |
| System Default | |
| /etc/ansible/hosts | |
| # static inventory | |
| inventory/ | |
| ├── hosts.ini # (hosts.yml) main inventory file | |
| ├── group_vars/ | |
| │ ├── all.yml # Variables applied to EVERY host | |
| │ ├── webservers.yml # Variables applied only to the 'webservers' group | |
| │ └── production.yml # Variables applied to the 'production' meta-group | |
| └── host_vars/ | |
| ├── web01.example.com.yml # Variables applied ONLY to web01 | |
| └── db01.example.com.yml # Variables applied ONLY to db01 | |
| # dynamic innvetory | |
| + GCP | |
| https://docs.ansible.com/projects/ansible/latest/collections/google/cloud/gcp_compute_inventory.html | |
| + AWS | |
| https://docs.ansible.com/projects/ansible/latest/collections/amazon/aws/docsite/aws_ec2_guide.html | |
| https://docs.ansible.com/projects/ansible/latest/collections/amazon/aws/aws_ec2_inventory.html | |
| # INI format (inventory.ini) | |
| # 1. UNGROUPED HOSTS | |
| standalone.example.com | |
| 192.168.1.100 | |
| # 2. HOST ALIASES & INLINE HOST VARIABLES | |
| [bastion] | |
| jumpbox ansible_host=10.0.0.5 ansible_user=ec2-user ansible_port=2222 | |
| # 3. RANGES (NUMERIC & ALPHABETIC) | |
| [dbservers] | |
| db-[01:03].example.com | |
| [cache] | |
| redis-[a:c].internal | |
| # 4. GROUP VARIABLES | |
| [webservers:vars] | |
| http_port=80 | |
| nginx_worker_processes=4 | |
| ansible_connection=ssh | |
| # 6. NESTED GROUPS (CHILDREN) (meta-groups) | |
| [production:children] | |
| webservers | |
| dbservers | |
| cache | |
| # 7. VARIABLES FOR NESTED GROUPS | |
| [production:vars] | |
| environment_tier=production | |
| monitoring_enabled=true | |
| api_endpoint=https://api.example.com | |
| # inventory YAML format (inventory.yml) | |
| --- | |
| ungrouped: | |
| hosts: | |
| mail.example.com: | |
| webservers: | |
| hosts: | |
| foo.example.com: | |
| www[01:50].example.com: | |
| dbservers: | |
| hosts: | |
| one.example.com: | |
| two.example.com: | |
| three.example.com: | |
| prod: | |
| children: | |
| dbservers: | |
| ... | |
| --- ROLE --- | |
| # role directory structure | |
| role_name/ | |
| ├── README.md | |
| ├── defaults/ | |
| │ └── main.yml | |
| ├── files/ | |
| ├── handlers/ | |
| │ └── main.yml | |
| ├── meta/ | |
| │ └── main.yml | |
| ├── tasks/ | |
| │ └── main.yml | |
| │ └── task-1.yml | |
| │ └── task-2.yml | |
| ├── templates/ | |
| ├── tests/ | |
| │ ├── inventory | |
| │ └── test.yml | |
| └── vars/ | |
| └── main.yml | |
| # create role | |
| --- TEST/CHECK --- | |
| # dry run | |
| $ ansible-playbook playbook.yml --check --diff | |
| # Syntax Validation | |
| $ ansible-playbook playbook.yml --syntax-check | |
| # Connectivity & Inventory Testing | |
| Ping all hosts | |
| $ ansible all -m ping | |
| View dynamic/static inventory | |
| $ ansible-inventory --graph | |
| # Code Quality & Best Practices (Linting) | |
| $ ansible-lint playbook.yml | |
| # Advanced Role Testing (Molecule) | |
| $ molecule test | |
| --- VARIABLES | |
| # variable precedence/priority/hierarchy | |
| Command-line values (for example, -u my_user, these are not variables) | |
| Role defaults (as defined in Role directory structure) | |
| Inventory file or script group vars | |
| Inventory group_vars/all | |
| Playbook group_vars/all | |
| Inventory group_vars/* | |
| Playbook group_vars/* | |
| Inventory file or script host vars | |
| Inventory host_vars/* | |
| Playbook host_vars/* | |
| Host facts and cached set_facts | |
| Play vars | |
| Play vars_prompt | |
| Play vars_files | |
| Role vars (as defined in Role directory structure) | |
| Block vars (for tasks in block only) | |
| Task vars (for the task only) | |
| include_vars | |
| Registered vars and set_facts | |
| Role (and include_role) params | |
| include params | |
| Extra vars (for example, -e "user=my_user")(always win precedence) | |
| # special env vars | |
| ANSIBLE_VERBOSITY=[1-5] | |
| ANSIBLE_DEBUG=[1|0] | |
| --- ISSUE --- | |
| ## SHORT-SHA miss-interpreted | |
| https://stackoverflow.com/questions/59066160/issue-deploying-to-appengine-flex-from-cloud-build | |
| --- PLUGINS --- | |
| ## gcp plugin for ansible | |
| --> inventory.gcp.yml | |
| plugin: gcp_compute | |
| projects: | |
| - gcp_project | |
| auth_kind: serviceaccount | |
| service_account_file: '/path/to/service-account.json' | |
| --> ansible.cfg | |
| [inventory] | |
| enable_plugins = gcp_compute | |
| --- BLOCK --- | |
| - name: Multiple task in a block | |
| block: | |
| - name: 1st task | |
| module: | |
| param: value | |
| - name: 2nd task | |
| module: | |
| param: value | |
| --- SERVICES ---- | |
| - name: Get service facts & check if a service exist | |
| block: | |
| - name: Get services facts | |
| service_facts: | |
| - name: Check if my_service is installed | |
| fail: | |
| msg: my_service is not installed, install it before | |
| when: ansible_facts.services["my_service.service"] is not defined | |
| --- COLLECTION --- | |
| ## install collection | |
| $ ansible-galaxy collection install --collections-path ~/.ansible/collections ansible.posix | |
| requirements.yml | |
| ``` | |
| collections: | |
| - community.general | |
| - ansible.posix | |
| ``` | |
| $ ansible-galaxy collection install --force --collections-path /root/.ansible/collections -r requirements.yml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## disable directory listing | |
| <Directory /var/www/public_html> | |
| Options -Indexes | |
| </Directory> | |
| ## force redirect non-www to https://wwww | |
| RewriteEngine On | |
| RewriteCond %{HTTP_HOST} !^www\. [NC] | |
| RewriteRule ^(.*)$ https://www.%{HTTP_HOST}%{REQUEST_URI} [R=301,L] | |
| ## force redirect http|www|non-www to https|non-www | |
| RewriteEngine On | |
| RewriteCond %{SERVER_NAME} =DOMAIN.COM [OR] | |
| RewriteCond %{SERVER_NAME} =WWW.DOMAIN.COM | |
| RewriteRule ^ https://DOMAIN.COM%{REQUEST_URI} [END,NE,R=permanent] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # API architecture type | |
| + REST (Representational State Transfer) |json, xml | |
| HTTP methods (get/post/put/delete) | |
| + GraphQL |json | |
| specific query for exact data to be retrieved or modified | |
| + gRPC (Google Remote Procedure Call) |Protocol Buffers (Binary) | |
| HTTP/2, Protocol Buffers | |
| call method on remote server as local object | |
| + SOAP (Simple Object Access Protocol) |xml | |
| structured, legacy protocol | |
| + WebSockets/Event-Driven APIs |json, binary | |
| create a persistent, two-way connection | |
| server auto pushes updates to client at event | |
| # API scope/policy | |
| - Public (Open) | |
| - Internal (Private) | |
| - Partners | |
| - Composites |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Domain Driven Design (DDD) | |
| Loose coupling and API-driven architecture | |
| Cloud-native architecture | |
| Microservices and containerization | |
| Serverless architecture | |
| Strangler pattern | |
| --- MISC | |
| # Sync VS Async | |
| + Sync | |
| - API (REST, GraphQL )with real-time process | |
| Cloud Run, App Engine, GKE | |
| - Routing HTTP request to backend | |
| API Gateway,Cloud LB | |
| - Instant database ACID | |
| Cloud SQL, Firestore | |
| + Async | |
| - event-driven message queue | |
| Pub/Sub | |
| - task schedule | |
| Cloud Task | |
| - event-driven task | |
| Eventarc | |
| # stateful VS stateless | |
| + stateful | |
| hold data locally | |
| + stateless | |
| new environment each time, no data hold | |
| # mutable VS immutable | |
| + mutable | |
| updated and modified in place after it is deployed | |
| + immutable | |
| never changed when deployed, | |
| for update, old version destroyed and replaced | |
| # imperative VS declarative | |
| + imperative | |
| exact step-by-step commands | |
| + declarative | |
| set the desired final state | |
| # monolyth VS microservice | |
| + monolyth | |
| entire application as one massive & unified codebase | |
| + microservice | |
| small, independent service that communicate over network | |
| # Scale Up VS Scale Out | |
| + Scale Up (vertical) | |
| add ressources to server | |
| + Scale Out (horizontal) | |
| add more identical servers | |
| # Push VS Pull | |
| + Push | |
| central system launch update servers | |
| + Pull | |
| servers look for update from central | |
| --- ARCHITECTURE DESIGN --- | |
| # Hub & Spoke | |
| + basics | |
| - Hub VPC: central VPC hosts shared services: | |
| . Connectivity: VPN Gateways or Interconnects to on-premise data centers. | |
| . Security: centralized Firewalls (NVA - Network Virtual Appliances) or Cloud Armor policies. | |
| . Tooling: CI/CD pipelines, bastion hosts, or shared logging/monitoring endpoints. | |
| - Spoke VPCs: | |
| Separate VPCs for different business units/ environments (Dev, Test, Prod) | |
| Hold application workloads | |
| + The Connectivity Problem (The "Gotcha") | |
| VPC Peering is NOT transitive. | |
| Spoke A can't talk to Spoke B or On-Prem via the Hub | |
| + solution | |
| . manual | |
| deploy NVA (Network Virtual Appliances) with Virtual routers/firewalls (Fortigate, Cisco VM) | |
| in Hub VPC to route traffic between spokes | |
| . Network Connectivity Center (NCC) | |
| managed service to automate routing between different networks | |
| Acts as"Hub", attach "Spokes" and handles the connectivity | |
| # Strangler Fig pattern | |
| cloud migration strategy/archi | |
| FOR re-archi/modernize monolithic app by gradually replace functionalities to new microservices | |
| --- APP ARCHITECTURE | |
| # SAGA pattern | |
| handle transactions across multiple db in microservices archi | |
| $ rollback others services (COMPENSATION) when last service fail | |
| EX: order OK, payment OK, stock KO | |
| -> cancel order, refund, update stock | |
| # microservice app archi | |
| + Sidecar Pattern | |
| helper service (network, logging) aside core service | |
| $ ISTIO (service mesh) | |
| + BFF (Backend For Frontend) | |
| custom backend built only for one client type | |
| $ bff for iOS, Android, Web App | |
| + Circuit Breaker | |
| fault-tolerance | |
| $ temporarily block requests to failing service | |
| + Database per Service | |
| each microservice owns its own database |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| # usage | |
| - k8s GitOps operator | |
| - sync k8s cluster with git repo | |
| # Deployment Architecture | |
| + Standalone | |
| + Hub & Spoke | |
| - 1 central cluster (ArgoCD) | |
| - X apps cluster | |
| + Decoupled Control Plane / Agent-based pull model | |
| - Central Control Plane (HUB) | |
| Web UI, API Server, Repo Server (clones repo), Dex (SSO) | |
| - Data Planes (SPOKES) | |
| agent with ArgoCD Application Controller in remote clusters | |
| # components | |
| + argocd-server | |
| API + web interface | |
| + repo-server | |
| git clone + final manifest YAML | |
| + application-controller< | |
| git/cluster synchro | |
| + Redis | |
| state caching | |
| + Dex | |
| SSO/OIDC | |
| # control loop (modele de reconciliation) - every 3min /git webhook | |
| 1. OBSERVE (live state) | |
| 2. DIFF (desired state) | |
| 3. ACT | |
| 4. REPEAT | |
| # sync status | |
| - Synced | |
| - OutOfSync | |
| - Unknown | |
| # health status | |
| - Healthy | |
| - Progressing | |
| - Degraded | |
| - Suspended | |
| - Missing | |
| - Unknown | |
| --- PROCESS | |
| # sync-phase | |
| 1: PreSync | |
| - configMap | |
| - job | |
| 2: Sync [default] | |
| - deployment | |
| 3: Post-Sync | |
| - job | |
| (FAIL) -> SyncFail | |
| - hook | |
| ... | |
| annotations: | |
| argocd.argoproj.io/hook: PreSync | |
| argocd.argoproj.io/hookdeletepolicies: x | |
| ... | |
| # sync-waves | |
| annotation to set a wave number to CRD | |
| FOR deployment with dependencies | |
| (wave 0) | |
| apiVersion: argoproj.io/v1alpha1 | |
| kind: Application | |
| metadata: | |
| name: kyverno | |
| annotations: | |
| argocd.argoproj.io/sync-wave: "1" | |
| (wave 2) | |
| apiVersion: argoproj.io/v1alpha1 | |
| kind: Application | |
| metadata: | |
| name: kyverno-policies | |
| annotations: | |
| argocd.argoproj.io/sync-wave: "2" | |
| # ignoreDifferences | |
| - hpa | |
| - istio (container side-car) | |
| - cert-manager | |
| - external-secret | |
| ... | |
| spec: | |
| ignoreDifferences: | |
| - group: apps | |
| kind: Deployment | |
| jsonPointers: - /spec/replicas | |
| ... | |
| # ACTION | |
| + Refresh/ Hard refresh | |
| force check repo/ invalidate cache | |
| + Prune | |
| delete unspecified ressouces | |
| + Dry run | |
| sync no apply ->TEST hook, waves | |
| + Apply only | |
| only deploy SYNC phase (ignore PreSync/PostSync) | |
| + Force | |
| delete, re-create -> DOWNTIME | |
| # ANNOTATION | |
| + Skip schema validation | |
| skip kubectl schema check | |
| + Prune last | |
| prune only at end | |
| + Respect ignoreDifferences | |
| use the annotation | |
| + Auto-create namespace | |
| if not exist | |
| + Apply out-of-sync only | |
| only these ressouces | |
| + SSA (Server Side Apply) | |
| prevent multiple overwrite (for ignoreDifferences) | |
| + Prune propagation policy | |
| - foreground -> child, parent | |
| - background -> parent, child | |
| - orphan -> parent only | |
| + Replace | |
| use kubectl replace instead of apply -> NO ROLLING UPDATE | |
| + Retry | |
| re-sync when fail -> succeed/ retry backoff/ retry limit | |
| + AllowEmpty | |
| no prune ressouces when repo empty | |
| + SelfHeal | |
| auto-sync when drift | |
| --- CRD/ K8S OBJECT | |
| # application | |
| - from where (git repo, dir, revision) | |
| - where to deploy (cluster, namespace) | |
| - sync type (manual, auto) | |
| apiVersion: argoproj.io/v1alpha1 | |
| kind: Application | |
| metadata: | |
| name: app-name | |
| namespace: argocd | |
| spec: | |
| source: | |
| repoURL: https://URL/repo | |
| targetRevision: main | |
| path: app/overlays/ENV | |
| destination: | |
| server: https://kubernetes.default.svc | |
| namespace: ENV | |
| syncPolicy: | |
| automated: | |
| selfHeal: true | |
| prune: true | |
| # appProject | |
| security check | |
| - who can deploy what, where? | |
| apiVersion: argoproj.io/v1alpha1 | |
| kind: AppProject | |
| metadata: | |
| name: equipe-backend | |
| namespace: argocd | |
| spec: | |
| sourceRepos: | |
| - https://github.com/mon-org/config-repo | |
| destinations: | |
| - namespace: backend-* | |
| server: https://kubernetes.default.svc | |
| clusterResourceWhitelist: | |
| - group: '' | |
| kind: Namespace | |
| # applicationSet | |
| code template to generate application | |
| - deploy to multiple namespaces | |
| - deploy to multiple clusters | |
| - change only values | |
| - change versions | |
| + applicationSet sources (Generators) | |
| - k8s cluster list | |
| - directory in git repo (./apps/chars/*) | |
| - git repo PR | |
| - external API (ticket manager) | |
| + Generators | |
| - list | |
| - cluster | |
| - git | |
| - matrix | |
| - merge | |
| - SCM provider | |
| - Pull Request | |
| - Cluster Decision | |
| - plugin | |
| apiVersion: argoproj.io/v1alpha1 | |
| kind: ApplicationSet | |
| metadata: | |
| name: apps-by-env | |
| namespace: argocd | |
| spec: | |
| generators: | |
| - list: | |
| elements: | |
| - env: dev | |
| namespace: app-dev | |
| - env: staging | |
| namespace: app-staging | |
| - env: production | |
| namespace: app-production | |
| template: | |
| metadata: | |
| name: 'mon-app-{{env}}' | |
| spec: | |
| source: | |
| repoURL: https://github.com/mon-org/config-repo | |
| path: 'mon-app/overlays/{{env}}' | |
| targetRevision: main | |
| destination: | |
| server: https://kubernetes.default.svc | |
| namespace: '{{namespace}}' | |
| syncPolicy: | |
| automated: | |
| selfHeal: true | |
| prune: true | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| # use-case | |
| + Compute | |
| - VM: Compute Engine Amazon EC2 | |
| - Auto-scaled VM: MIG Amazon EC2 Auto Scaling | |
| - VMWare: VMWare Engine VMware Cloud on AWS | |
| + Containers | |
| - app only: App Engine AWS Elastic Beanstalk | |
| - container/image: Cloud Run AWS Fargate / Amazon ECS | |
| - orchestrator: GKE Amazon EKS | |
| - registry: Artifact Registry Amazon ECR | |
| + Database | |
| - SQL: CloudSQL -> Cloud Spanner Amazon RDS -> Amazon Aurora | |
| - NoSQL: Datastore -> BigTable (Time) Amazon DynamoDB -> Amazon Timestream (Time) | |
| - Live/real-time mobile/IoT: Firestore AWS AppSync / DynamoDB | |
| - App cache: Memorystore Amazon ElastiCache | |
| - Object: Cloud Storage Amazon S3 | |
| - Disk: Persistent Disk -> Filestore Amazon EBS -> Amazon EFS | |
| + Network | |
| - Private network/ 1 project: VPC, Subnet Amazon VPC, Subnet | |
| - 1-for-all VPC: Shared VPC AWS Transit Gateway / VPC Sharing | |
| - Interco VPC: VPC Peering -> (mesh) NCC VPC Peering -> (mesh) AWS Transit Gateway | |
| - Intern-only access to gAPI: PGA -> (endpoint) PSC VPC Endpoints -> (endpoint) AWS PrivateLink | |
| - Access to Tenant-VPC: PSA -> (PGA+PSA) PSC AWS PrivateLink / VPC Peering | |
| - Intern endpoint: PSC VPC Endpoint (AWS PrivateLink) | |
| - Load balancer: Cloud LB Elastic Load Balancing (ALB/NLB) | |
| - inter-cloud/on-prem: Cloud Interconnect AWS Direct Connect | |
| - VPN: Cloud VPN AWS Site-to-Site VPN | |
| - CDN: Cloud CDN Amazon CloudFront | |
| - NAT: Cloud NAT NAT Gateway | |
| - DNS: Cloud DNS Amazon Route 53 | |
| - IP: External IP/ Private IP Elastic IP / Private IP | |
| + Dev | |
| - CI/CD: Cloud Build AWS CodePipeline / CodeBuild | |
| - IaC: Deployment Manager AWS CloudFormation | |
| - API endpoint: API Gateway Amazon API Gateway | |
| - API management: Apigee Amazon API Gateway | |
| - function only: Cloud Function AWS Lambda | |
| - Message queue: Pub/Sub Amazon SQS / Amazon SNS | |
| - Event-driven task: Eventarc Amazon EventBridge | |
| + Security | |
| - Firewall (L3-4, VM): Firewall rules Security Groups (SG) / Network ACLs | |
| - WAF (L7, LB): Cloud Armor AWS WAF | |
| - Perimeter isolation: VPC-SC AWS PrivateLink / SCPs | |
| + IAM | |
| - IDaaS + Import Directory: Cloud Identity AWS IAM Identity Center | |
| - RBAC + manage user: IAM + Service Account (GSA) AWS IAM (Users + Roles) | |
| - GKE IAM: Workload Identity (GSA + KSA) IAM Roles for Service Accounts (IRSA) | |
| - Principle: At-least Priv(3W) + Zero-Trust Least Privilege + Zero Trust | |
| - Keys management: Cloud KMS AWS KMS | |
| + BigData/Analytics | |
| - Data warehouse: BigQuery Amazon Redshift | |
| - Workflow orchestrator: Cloud Composer (Apache Airflow) Amazon MWAA (Apache Airflow) | |
| - Stream/batch process: DataFlow (Apache Beam) Amazon Kinesis / AWS Glue | |
| - Big data ecosystem: DataProc (Hadoop, Spark) Amazon EMR | |
| - No-code/UI ETL: Data Fusion AWS Glue Studio | |
| - BI platform: Looker Amazon QuickSight | |
| + Observability | |
| - Log: Cloud Logging Amazon CloudWatch Logs | |
| - Monitoring: Cloud Monitoring Amazon CloudWatch | |
| + AI | |
| - Generative AI: Vertex AI Studio + Gemini Amazon Bedrock | |
| - MLOps: Vertex AI Platform Amazon SageMaker | |
| - Media analyze: Vision AI Amazon Rekognition | |
| - Text Translation: Cloud Nat. Language & Trans. Amazon Comprehend & Amazon Translate | |
| --- INSTANCE | |
| # start/stop instance | |
| Get instance ID | |
| $ INSTANCE_ID=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=xfusion-ec2" --query "Reservations[0].Instances[0].InstanceId" --output text) | |
| Stop | |
| $ aws ec2 stop-instances --instance-ids $INSTANCE_ID | |
| Start | |
| $ aws ec2 start-instances --instance-ids $INSTANCE_ID | |
| Check status | |
| $ aws ec2 describe-instances --instance-ids $INSTANCE_ID --query "Reservations[].Instances[].{ID: InstanceId, State: State.Name}" | |
| # Edit instance attribute | |
| $ aws ec2 modify-instance-attribute \ | |
| --instance-id $INSTANCE_ID \ | |
| --instance-type "{\"Value\": \"NEW_VALUE\"}" | |
| # create instance ec2 | |
| Get image ID | |
| $ AMI_ID=$(aws ssm get-parameters --names /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 --query 'Parameters[0].Value' --output text) | |
| Get default Security Group ID | |
| $ SG_ID=$(aws ec2 describe-security-groups --filters "Name=group-name,Values=default" --query "SecurityGroups[0].GroupId" --output text) | |
| Create instance | |
| $ aws ec2 run-instances \ | |
| --image-id $AMI_ID \ | |
| --instance-type t2.micro \ | |
| --key-name MY_INSTANCE \ | |
| --security-group-ids $SG_ID \ | |
| --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=MY_INSTANCE}]' | |
| --- IP/Adresses | |
| # allocate Elastic IP (EIP) | |
| $ aws ec2 allocate-address | |
| --domain vpc | |
| --tag-specifications 'ResourceType=elastic-ip,Tags=[{Key=Name,Value=MY_EIP}]' | |
| # list addresses | |
| $ aws ec2 describe-addresses | |
| --query 'Addresses[].{IP:PublicIp, ID:AllocationId, Instance:InstanceId}' --output table | |
| --filters "Name=tag:Name,Values=MY_EIP" | |
| # attach IP to instance | |
| $ aws ec2 associate-address \ | |
| --allocation-id ID_EIP \ | |
| --instance-id ID_INSTANCE | |
| --- KEYS | |
| # create EC2 key pair --key-name devops-kp --key-type rsa --query 'KeyMaterial' | |
| --- VOLUME | |
| # create ec2 volume | |
| $ aws ec2 create-volume | |
| --volume-type gp3 --size 2 | |
| --tag-specifications 'ResourceType=volume,Tags=[{Key=Name,Value=datacenter-volume}]' | |
| # list ec2 volumes | |
| $ aws ec2 describe-volumes | |
| --filters Name=status,Values=available | |
| --query 'Volumes[*].{ID:ID_VOLUME,Size:SIZE}' | |
| --output table |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| # use-case | |
| + Compute | |
| - VM: Compute Engine AZ Virtual Machines | |
| - Auto-scaled VM: MIG VMSS (VM Scale Sets) | |
| - VMWare: VMWare Engine AZ VMware Solution | |
| + Containers | |
| - app only: App Engine AZ App Service | |
| - container/image: Cloud Run AZ Container Apps | |
| - orchestrator: GKE AKS (AZ Kub Service) | |
| - registry: Artifact Registry AZ Container Registry | |
| + Database | |
| - SQL: CloudSQL -> Cloud Spanner AZ SQL Database for MySQL/PostgreSQL | |
| - NoSQL: Datastore -> BigTable (Time) AZ Cosmos DB | |
| - Live/real-time mobile/IoT: Firestore AZ Cosmos DB | |
| - App cache: Memorystore AZ Cache for Redis | |
| - Object: Cloud Storage AZ Blob Storage | |
| - Disk: Persistent disk AZ Managed Disks | |
| - NFS: Filestore AZ Files | |
| + Network | |
| - Private network/ 1 project: VPC, Subnet VNet (AZ Virtual Network), Subnet | |
| - 1-for-all VPC: Shared VPC AZ Virtual WAN | |
| - Interco VPC: VPC Peering -> (mesh) NCC VNet Peering -> (mesh) AZ Virtual WAN | |
| - Intern-only access to gAPI: PGA -> (endpoint) PSC Service Endpoints -> (endpoint) AZ Private Link | |
| - Access to Tenant-VPC: PSA -> (PGA+PSA) PSC VNet Integration -> (PGA+PSA) AZ Private Link | |
| - Intern endpoint: PSC AZ Private Endpoint | |
| - Load balancer: Cloud LB AZ Load Balancer | |
| - inter-cloud/on-prem: Cloud Interconnect AZ ExpressRoute | |
| - VPN: Cloud VPN AZ VPN Gateway | |
| - CDN: Cloud CDN AZ Front Door / AZ CDN | |
| - NAT: Cloud NAT AZ NAT Gateway | |
| - DNS: Cloud DNS AZ DNS | |
| - IP: External IP/ Private IP Public IP / Private IP | |
| + Dev | |
| - CI/CD: Cloud Build AZ DevOps (Pipelines) | |
| - IaC: Deployment Manager AZ Resource Manager (ARM) / Bicep | |
| - API endpoint: API Gateway AZ API Management | |
| - API management: Apigee AZ API Management | |
| - function only: Cloud Function AZ Functions | |
| - Message queue: Pub/Sub AZ Service Bus | |
| - Event-driven task: Eventarc AZ Event Grid | |
| + Security | |
| - Firewall (L3-4, VM): Firewall rules Network Security Group (NSG) | |
| - WAF (L7, LB): Cloud Armor AZ Web Application Firewall (WAF) | |
| - Perimeter isolation: VPC-SC AZ Private Link | |
| + IAM | |
| - IDaaS + Import Directory: Cloud Identity Microsoft Entra ID | |
| - RBAC + manage user: IAM + Service Account (GSA) AZ RBAC + Managed Identities | |
| - GKE IAM: Workload Identity (GSA + KSA) Microsoft Entra Workload ID | |
| - Principle: At-least Priv(3W) + Zero-Trust Least Privilege + Zero Trust | |
| - Keys management: Cloud KMS AZ Key Vault | |
| + BigData/Analytics | |
| - Data warehouse: BigQuery AZ Synapse Analytics | |
| - Workflow orchestrator: Cloud Composer (Apache Airflow) AZ Data Factory | |
| - Stream/batch process: DataFlow (Apache Beam) AZ Databricks / Stream Analytics | |
| - Big data ecosystem: DataProc (Hadoop, Spark) AZ HDInsight | |
| - No-code/UI ETL: Data Fusion AZ Data Factory | |
| - BI platform: Looker Power BI | |
| + Observability | |
| - Log: Cloud Logging AZ Monitor Logs | |
| - Monitoring: Cloud Monitoring AZ Monitor | |
| + AI | |
| - Generative AI: Vertex AI Studio + Gemini AZ OpenAI Service | |
| - MLOps: Vertex AI Platform AZ Machine Learning | |
| - Media analyze: Vision AI AZ AI Video Indexer | |
| - Text Translation: Cloud Nat. Language & Trans. AZ AI Translator | |
| --- AZ SERVICES | |
| # Compute | |
| - Virtual Machines (VMs) | |
| @ VM, migrate on-prem server, custom soft | |
| - AKS (Azure Kubernetes Service) | |
| container orchestrator cluster | |
| @ manage containerized apps | |
| - Container Apps | |
| Serverless container platform, create your image/container, portable to k8s/KEDA | |
| @ host containerized app, auto-scale based on traffic | |
| - App Service | |
| PaaS, focus on app runner only | |
| @ build/host web app without managing infra | |
| - Azure Functions | |
| @ serverless code/batch runner | |
| - ACR (Azure Container Registry) / Azure Artifacts | |
| @ image/container & package registry | |
| - AVS (Azure VMware Solution) | |
| @ VMware workloads on Azure | |
| # Network | |
| - VNet (Virtual Network) | |
| private networking for cloud resources | |
| - Load Balancer / Application Gateway | |
| distributes incoming traffic across multiple backend servers | |
| @ high availability | |
| - Azure DNS | |
| DNS service (low latency/HA domain resolution) | |
| - NAT Gateway | |
| NAT gateway for private resources | |
| - Front Door / Azure CDN | |
| caches content close to users around the world | |
| @ reduce latency | |
| - ExpressRoute | |
| dedicated private connection between on-prem data center and Azure | |
| - VPN Gateway | |
| - Network Watcher | |
| network observability/monitoring | |
| - Route Server | |
| - Public IP address | |
| # Storage | |
| + Object storage | |
| - Blob Storage | |
| large object storage | |
| @ unstructured data: backups, archives, multimedia files, static website content | |
| + Disk | |
| - Managed Disks | |
| block storage | |
| @ attach to VMs as hard drives | |
| - Azure Files / Azure NetApp Files | |
| managed NAS (network-attached storage) | |
| @ traditional legacy apps, use SMB/NFS interface | |
| # Database | |
| + SQL/RDBMS | |
| - SQL Database / Database for MySQL & PostgreSQL | |
| relational database service (regional/managed) | |
| @ MySQL, PostgreSQL, SQL Server | |
| - Cosmos DB / SQL Database Hyperscale | |
| globally distributed, horizontally scalable relational/multimodel database | |
| @ massive, mission-critical app | |
| - Database for PostgreSQL (Flexible Server) | |
| PostgreSQL-compatible database with enterprise performance | |
| @ top-tier transactional and analytical enterprise workloads | |
| + NoSQL | |
| (semi/unstructured data (doc, key-value pairs) -> real-time app, IoT, user profiles) | |
| - Cosmos DB (NoSQL API) | |
| serverless/provisioned globally distributed document database | |
| @ live, real-time syncing in mobile, web, IoT app | |
| - Cosmos DB (Table/Cassandra API) / Managed Instance for Apache Cassandra | |
| high-perf NoSQL db (for analytics/ML) | |
| @ heavy analytical workloads and high-throughput data collection | |
| - Table Storage | |
| simple key-value/NoSQL database | |
| - Cache for Redis | |
| in-memory data store Redis | |
| @ sub-millisecond data access for caching and fast-retrieval app | |
| # Event process | |
| - Service Bus / Event Hubs | |
| real-time/async messaging and event streaming service | |
| @ connect app and streaming data | |
| - Event Grid | |
| event-driven reactive programming and routing | |
| # Analytics/ BigData | |
| - Stream Analytics / Synapse Data Flows | |
| serverless stream/batch processing engine | |
| @ stream/batch processing | |
| - HDInsight / Synapse Spark | |
| managed Hadoop/Spark big data platform | |
| @ migrate existing big data ecosystems | |
| - Synapse Analytics / Microsoft Fabric | |
| serverless, scalable enterprise data warehouse and lakehouse | |
| @ run blazing-fast SQL queries on massive datasets | |
| - Data Factory (Managed Airflow) | |
| managed Apache Airflow orchestration service | |
| @ orchestrate workflows via Python code | |
| - Power BI | |
| enterprise business intelligence (BI) platform | |
| @ create interactive dashboards, data apps, embedded analytics | |
| - Data Factory | |
| visual, no-code/low-code data integration and ETL/ELT tool | |
| # AI/ML | |
| - AI Foundry / Azure Machine Learning | |
| unified machine learning platform | |
| @ build, train, deploy custom ML models and generative AI app | |
| - Microsoft Copilot Studio / OpenAI Assistants | |
| unified space for building enterprise AI agents | |
| @ utilize generative AI models securely in organization | |
| - AI Language (Text Analytics) | |
| analyzes text using pre-built deep learning models | |
| - AI Vision / AI Video Indexer | |
| pre-trained machine learning models | |
| @ analyze, classify images/videos | |
| - AI Speech (Speech-to-Text / Text-to-Speech) | |
| AI-driven tools | |
| @ convert spoken audio into text and synthesize speech | |
| - Machine Learning | |
| - AI Vision | |
| - AI Speech | |
| - AI Translator | |
| - N-Series VMs (GPU instances for ML) | |
| # IoT (Internet Of Things) | |
| - IoT Hub | |
| managing and connecting IoT devices | |
| # Identity/Security | |
| - Microsoft Entra ID (formerly Azure AD) & Azure RBAC | |
| identity management and granular access control | |
| @ define who can take what action on specific cloud resources | |
| - Key Vault | |
| manage cryptographic keys, secrets, and certificates used for cloud services | |
| - DDoS Protection & Azure WAF (Web Application Firewall) | |
| DDoS protection and layer 7 web application firewall | |
| @ protect app from targeted threats | |
| - Network Security Groups (NSGs) / Firewall rules | |
| - Microsoft Defender for Cloud | |
| comprehensive security posture management and threat protection platform | |
| @ identify vulnerabilities/threats across cloud workloads | |
| # Developer & Management Tools | |
| - Azure Pipelines (Azure DevOps) / GitHub Actions | |
| continuous integration and continuous delivery (CI/CD) platform | |
| @ automates code deployment | |
| - ARM (Azure Resource Manager) Templates / Bicep | |
| native IaC (Infrastructure as Code) tools | |
| - API Management (Consumption Tier) | |
| serverless gateway for secure access to backend by REST API | |
| - API Management (APIM) | |
| full-lifecycle enterprise API management platform | |
| # Migration | |
| - AMMP (Azure Migration and Modernization Program) | |
| migration guidance, funding, and assessment program | |
| - Azure Migrate | |
| on-prem/multi-cloud discovery, assessment, and migration hub to Azure | |
| - Microsoft Cloud Adoption Framework for Azure (CAF) | |
| best-practice documentation and guidance for cloud adoption | |
| # Monitoring/Observability | |
| - Azure Monitor (Metrics) | |
| monitor performance metrics | |
| - Azure Monitor Logs (Log Analytics / Application Insights) | |
| capture logs (centralized log management and APM) | |
| # Command-Line tools | |
| - az (Azure CLI) | |
| - AzCopy / Azure Storage CLI | |
| - sqlcmd / mssql-cli | |
| - kubectl | |
| --- STORAGE | |
| # storage redundancy options (RS: Redundancy Storage) | |
| + Primary region | |
| - LRS (Locally RS) | |
| - ZRS (Zone RS) | |
| + Geo-Redundant region | |
| - GRS (Geo RS) | |
| - GZRS (Geo-Zone RS) | |
| + Read-Access Add-ons | |
| - RA-GRS / RA-GZRS |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| # Landing Zone | |
| (cloud infra foundation) | |
| workspace base to host cloud platform, to onboard multiple teams with established configs | |
| - hierarchy structure | |
| - IAM/roles/permissions | |
| - group management | |
| - network access/policies, | |
| - ... | |
| + Type | |
| - LZ platform | |
| for cloud users using cross-project services | |
| . network, interconnect/vpn (on-prem, ext cloud) | |
| . DNS, CDN | |
| . IAM, MFA | |
| . Infra monitoring | |
| . billing/ finOps | |
| - LZ app | |
| . subnet | |
| . backup | |
| . IAM | |
| + Process | |
| - define cloud strategy | |
| - goal alignment plan | |
| - landing zone | |
| - modernization method | |
| - IT gouvernance | |
| + GCP LZ | |
| Prerequisite to deploy enterprise workloads in GCP | |
| -> https://docs.cloud.google.com/architecture/landing-zones | |
| - Resource Hierarchy | |
| . org node | |
| . folder | |
| . project | |
| . ressource | |
| - IAM | |
| . Cloud identity Directory | |
| . roles/policies | |
| - Network | |
| . VPV, Shared VPC, VPC Peering | |
| . External interconnect (VPN) | |
| . Network policies (firewall, cloud armor) | |
| - Monitoring | |
| - Billing/ FinOps | |
| - IaC | |
| - Gouvernance/ Compliance | |
| # CLOUD services matching, use case | |
| + Compute | |
| - VM: Compute Engine Amazon EC2 AZ Virtual Machines | |
| - Auto-scaled VM: MIG Amazon EC2 Auto Scaling VMSS (VM Scale Sets) | |
| - VMWare: VMWare Engine VMware Cloud on AWS AZ VMware Solution | |
| + Containers | |
| - app only: App Engine AWS Elastic Beanstalk AZ App Service | |
| - container/image: Cloud Run AWS Fargate / Amazon ECS AZ Container Apps | |
| - orchestrator: GKE Amazon EKS AKS (AZ Kub Service) | |
| - registry: Artifact Registry Amazon ECR AZ Container Registry | |
| + Database | |
| - SQL: CloudSQL -> Cloud Spanner Amazon RDS -> Amazon Aurora AZ SQL DB -> Cosmos DB (PostgreSQL) | |
| - NoSQL: Datastore -> BigTable (Time) Amazon DynamoDB -> Timestream AZ Cosmos DB -> Data Explorer | |
| - Live/real-time mobile/IoT: Firestore AWS AppSync / DynamoDB AZ Cosmos DB | |
| - App cache: Memorystore Amazon ElastiCache AZ Cache for Redis | |
| - Object: Cloud Storage Amazon S3 AZ Blob Storage | |
| - Disk: Persistent Disk -> Filestore Amazon EBS -> Amazon EFS AZ Managed Disks -> AZ Files | |
| + Network | |
| - Private network/ 1 project: VPC, Subnet Amazon VPC, Subnet VNet (AZ Virtual Network), Subnet | |
| - 1-for-all VPC: Shared VPC AWS Transit Gateway AZ Virtual WAN | |
| - Interco VPC: VPC Peering -> (mesh) NCC VPC Peering -> Transit Gateway VNet Peering -> AZ Virtual WAN | |
| - Intern-only access to gAPI: PGA -> (endpoint) PSC VPC Endpoints -> PrivateLink Service Endpoints -> Private Link | |
| - Access to Tenant-VPC: PSA -> (PGA+PSA) PSC AWS PrivateLink / VPC Peering VNet Integration -> Private Link | |
| - Intern endpoint: PSC VPC Endpoint (PrivateLink) AZ Private Endpoint | |
| - Load balancer: Cloud LB Elastic Load Balancing AZ Load Balancer | |
| - inter-cloud/on-prem: Cloud Interconnect AWS Direct Connect AZ ExpressRoute | |
| - VPN: Cloud VPN AWS Site-to-Site VPN AZ VPN Gateway | |
| - CDN: Cloud CDN Amazon CloudFront AZ Front Door / AZ CDN | |
| - NAT: Cloud NAT NAT Gateway AZ NAT Gateway | |
| - DNS: Cloud DNS Amazon Route 53 AZ DNS | |
| - IP: External IP/ Private IP Elastic IP / Private IP Public IP / Private IP | |
| + Dev | |
| - CI/CD: Cloud Build AWS CodePipeline / CodeBuild AZ DevOps (Pipelines) | |
| - IaC: Deployment Manager AWS CloudFormation ARM (AZ Resource Manager) / AZ Bicep | |
| - API endpoint: API Gateway Amazon API Gateway AZ API Management | |
| - API management: Apigee Amazon API Gateway AZ API Management | |
| - function only: Cloud Function AWS Lambda AZ Functions | |
| - Message queue: Pub/Sub Amazon SQS / Amazon SNS AZ Service Bus | |
| - Event-driven task: Eventarc Amazon EventBridge AZ Event Grid | |
| + Security | |
| - Firewall (L3-4, VM): Firewall rules Security Groups (SG) / ACLs NSG (Network Security Group) | |
| - WAF (L7, LB): Cloud Armor AWS WAF AZ WAF | |
| - Perimeter isolation: VPC-SC AWS PrivateLink / SCPs AZ Private Link | |
| + IAM | |
| - IDaaS + Import Directory: Cloud Identity AWS IAM Identity Center Ms Entra ID | |
| - RBAC + manage user: IAM + Service Account (GSA) AWS IAM (Users + Roles) AZ RBAC + Managed Identities | |
| - GKE IAM: Workload Identity (GSA + KSA) IAM Roles for Service Accounts Microsoft Entra Workload ID | |
| - Principle: At-least Priv(3W) + Zero-Trust Least Privilege + Zero Trust Least Privilege + Zero Trust | |
| - Keys management: Cloud KMS AWS KMS AZ Key Vault | |
| + BigData/Analytics | |
| - Data warehouse: BigQuery Amazon Redshift AZ Synapse Analytics | |
| - Workflow orchestrator: Cloud Composer Amazon MWAA AZ Data Factory | |
| - Stream/batch process: DataFlow Amazon Kinesis / AWS Glue AZ Databricks / Stream Analytics | |
| - Big data ecosystem: DataProc Amazon EMR AZ HDInsight | |
| - No-code/UI ETL: Data Fusion AWS Glue Studio AZ Data Factory | |
| - BI platform: Looker Amazon QuickSight Power BI | |
| + Observability | |
| - Log: Cloud Logging Amazon CloudWatch Logs AZ Monitor Logs | |
| - Monitoring: Cloud Monitoring Amazon CloudWatch AZ Monitor | |
| + AI | |
| - Generative AI: Vertex AI Studio + Gemini Amazon Bedrock AZ OpenAI Service | |
| - MLOps: Vertex AI Platform Amazon SageMaker AZ Machine Learning | |
| - Media analyze: Vision AI Amazon Rekognition AZ AI Video Indexer | |
| - Text Translation: Cloud Nat. Language & Trans. Amazon Comprehend & Translate AZ AI Translator | |
| --- CLOUD MIGRATION | |
| (Azure) | |
| # CLOUD ADOPTION FRAMEWORK | |
| +1: Assessment (Where are they now?) | |
| $ assess CAF theme | |
| - LEARN (Skills) | |
| "Only two engineers have touched GCP. There is no official training budget" | |
| - LEAD (Culture) | |
| "The CIO is championing the cloud move, but middle management is resisting the change" | |
| - SCALE (Processes) | |
| Infrastructure is built manually. No automation or code. | |
| - SECURE (Risk) | |
| Security relies on a perimeter firewall. No identity management/least-privilege access rules | |
| +2: Define Goal (Where do they want to be?) | |
| Realistic goal for next 12m to get to Strategic phase | |
| Ex: safely migrate e-commerce platform on cloud | |
| +3: Action Plan (EPICS) | |
| $ set CAF theme | |
| - LEARN -> Upskill the team | |
| experienced mentor the rest of the staff | |
| internal workshops | |
| - LEAD -> Align to business | |
| multiple teams (Network, Storage) to cross-functional Product Teams | |
| FinOps culture | |
| Business KPIs (e.g., "Reduce website load time by 20%" rather than just "Move servers"). | |
| - SCALE -> Automate operations | |
| use IaC (terraform) | |
| CI/CD pipeline | |
| - SECURE -> Modernize protection | |
| Cloud IAM policy, least privilege | |
| # MOVE-TO-CLOUD strategies | |
| - Rehost/ Lift-&-Shift (IaaS) | |
| rehost app without architecture change | |
| - Refactor (PaaS) | |
| move app to cloud infrastructure | |
| - Rebuild | |
| throw out the code for an existing app and rearchitect it | |
| - Replace (SaaS) | |
| discard existing app and adopt commercial SaaS | |
| https://bluexp.netapp.com/blog/aws-cvo-blg-strategies-for-aws-migration-the-new-7th-r-explained | |
| https://drive.google.com/file/d/16TcsxMpjL0Uda_IUj6UF0RS55Q9bW0XG/view?usp=sharing | |
| + Gartner 5 Rs Model | |
| - Rehost (lift-and-shift) | |
| - Refactor (re-architecting) | |
| - Revise (part optimize apps for the cloud) | |
| - Rebuild (to cloud-native) | |
| - Replace | |
| + AWS 6 Rs Model | |
| - Rehosting (Lift-and-Shift) | |
| - Replatforming (Lift, Tinker, and Shift) (minor optimizations) | |
| - Repurchasing (Drop-and-Shop) (replace with SaaS) | |
| - Refactoring / Re-architecting | |
| - Retire (Shut down unnecessary apps) | |
| - Retain (Revisit) (Keep certain apps on-premises) | |
| (7) - Relocate | |
| Move entire data centers or large portions of infrastructure to the cloud with minimal changes | |
| --- CONCEPT | |
| # auto-* | |
| + auto-scaling | |
| adjusts resources based on load | |
| + auto-healing | |
| detects anomalies: config drifts, health checks, security issues | |
| --- SECURITY | |
| # API security | |
| - .env OUT OF REPO (.gitignore) | |
| - rate limit on login | |
| - RLS | |
| - session expire | |
| - input/file type check before process | |
| - upload max size | |
| - config CORS | |
| - signed webhooks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- RDBMS | |
| # tools | |
| - mysql/ mariadb | |
| - postgresql | |
| - SQL Server | |
| - oracle |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- deployment | |
| # deployment pattern | |
| + Canary | |
| to small % users | |
| + Dark Launch | |
| to hidden prod by feature flags | |
| + Blue/Green | |
| flip user traffic from old prod (Blue) to identical, fully updated parallel environment (Green) | |
| + Rolling Update | |
| replaces instances gradually | |
| + Shadow | |
| duplicates real user traffic without return results | |
| --- IAM | |
| # auth framework | |
| + OpenID Connect (OIDC) | |
| authentication | |
| @ verify identity | |
| + OAuth2 | |
| authorization | |
| @ allow access to user's resources (info, files, contacts) | |
| --- RELEASE MANAGEMENT --- | |
| # release strategy | |
| + Big Bang | |
| replace all with new at once - requiring downtime | |
| + Rolling | |
| updates gradually - no downtime, run both versions | |
| + Blue/Green | |
| uses two identical prod envs (live and idle) | |
| traffic switched to new env once verified | |
| + Canary | |
| releases new to few real users first to monitor for issues before rolling all out | |
| + Feature Toggles | |
| Deploys behind a configuration switch, | |
| allowing features to be turned on or off without deploying new code | |
| + Shadow | |
| duplicates live user traffic to the new version in the background to test performance | |
| then load without impacting the actual user experience. | |
| --- TEST | |
| # SAST (Static Application Security Testing) analysis | |
| source code analysis without exec | |
| $ detect vulnerabilities |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## reverse lookup | |
| $ dig -x IP_ADDRESS |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC --- | |
| ## install | |
| ## post install | |
| Non-root user access | |
| $ sudo groupadd docker | |
| $ sudo usermod -aG docker $USER | |
| --- BUILD --- | |
| # layers (build instruction) | |
| + physical | |
| FROM: base operating system/runtime | |
| RUN: execution, runs terminal (only on BUILD) | |
| COPY: copy from local to image | |
| ADD: advanced COPY, extract local archives, download from remote URLs. | |
| + logical | |
| WORKDIR: default work dir path | |
| ENV: env vars | |
| ARG: build-time vars | |
| ENTRYPOINT: hardcoded main executable for the container | |
| CMD: default command to run when container starts (on RUNTIME) (arg to entrypoint) | |
| EXPOSE: Documents which network ports the container intends to listen on. | |
| VOLUME: data in this directory should be persisted outside the container's lifecycle | |
| LABEL: custom metadata (version="1.0") | |
| USER: specific username or UID for RUN, CMD, or ENTRYPOINT | |
| HEALTHCHECK: test container if working (curl /health endpoint) | |
| SHELL: override default shell | |
| ONBUILD: | |
| ## build image | |
| $ docker build -t IMAGE_NAME:TAG . | |
| $ docker build -t IMAGE_NAME:TAG -f /PATH/TO/DOCKERFILE | |
| # multi-stage build image | |
| # (1) the builder | |
| FROM golang:1.22 AS builder | |
| WORKDIR /app | |
| COPY go.mod go.sum ./ | |
| RUN go mod download | |
| COPY . . | |
| RUN CGO_ENABLED=0 GOOS=linux go build -o mon-application . | |
| # (2) final image | |
| FROM gcr.io/distroless/static-debian12 | |
| WORKDIR /root/ | |
| COPY --from=builder /app/mon-application . | |
| CMD ["./mon-application"] | |
| --- FILE --- | |
| ## copy file | |
| + host to container | |
| $ docker cp FILE_ON_HOST CONTAINER_ID:FILE | |
| + container to host | |
| $ docker cp CONTAINER_ID:FILE FILE_ON_HOST | |
| --- IMAGE --- | |
| ## rename image | |
| $ docker tag CURRENT_IMAGE_NAME:TAG NEW_IMAGE_NAME:TAG | |
| --- LOGS | |
| # container logs | |
| $ docker logs CONTAINER (-f follow, --tail 50, --since 30m) | |
| --- RUN --- | |
| ## run container | |
| $ docker run --name CONTAINER_NAME -p LOCAL_PORT:CONTAINER_PORT -e ENV_VAR=VALUE IMAGE:TAG | |
| --- FILTER --- | |
| https://docs.docker.com/reference/cli/docker/image/ls/ | |
| ## KEYS | |
| id - container's ID | |
| name - container's name | |
| label - <key> or <key>=<value> | |
| exited - container's exit code (Only useful with --all) | |
| status - created, restarting, running, removing, paused, exited, or dead | |
| ancestor - image as an ancestor (image, image:tag, image:tag@digest, short-id, full-id) | |
| volume - given volume or bind mount | |
| network - given network | |
| publish/expose - given port: <port>[/<proto>] or <startport-endport>/[<proto>] | |
| health - starting, healthy, unhealthy or none. | |
| isolation - Windows daemon only: default, process, or hyperv. | |
| is-task - true or false | |
| $ docker ps --filter "key=value" --filter "key2=value2" | |
| --- FORMAT --- | |
| https://docs.docker.com/engine/cli/formatting/ | |
| ## COLUMN | |
| ID - container ID | |
| Image - image ID | |
| Command - quoted command | |
| CreatedAt - container creation time | |
| RunningFor - container running time | |
| Ports - exposed ports | |
| State - container status ("created", "running", "exited") | |
| Status - container status with details (duration, health-status) | |
| Size - container disk size | |
| Names - container names | |
| Labels - container's labels | |
| Label - container specific label. Ex: '{{.Label "com.docker.swarm.cpu"}}' | |
| Mounts - container's mounted volumes | |
| Networks - container's attached network | |
| ## inline | |
| $ docker ps --format "{{.COLUMN-1}}: {{.COLUMN-2}}" | |
| ## table | |
| $ docker ps --format "table {{.COLUMN-1}}\t{{.COLUMN-2}}\t{{.COLUMN-N}}" | |
| ## json | |
| $ docker ps --format json | |
| $ docker ps --format '{{json .COLUMN}}' | |
| --- DOCKER-COMPOSE --- | |
| ## exec | |
| $ docker-compose exec SERVICE cmd --arg param | |
| ## ERROR: Error creating container: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60) | |
| increase timeout | |
| $ COMPOSE_HTTP_TIMEOUT=200 docker-compose up | |
| ## USE YAML ANCHORS | |
| --> FUNCTION | |
| x-function: &function | |
| labels: | |
| my_label: "true" | |
| depends_on: | |
| - another_service | |
| networks: | |
| - my_network | |
| services: | |
| service-1: | |
| <<: *function | |
| image: dir/my_image_1:latest | |
| environment: | |
| my_env: "prod" | |
| service-2: | |
| <<: *function | |
| image: dir/my_image_2:health | |
| volume: | |
| my_volume:/opt/app | |
| --> ANCHORS | |
| --> FLAGS |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # parallelism | |
| - analyzing data, creating reports, and presenting findings | |
| - data analysis, reports creation, and findings presentation | |
| --- VOCABULARY | |
| # idiom | |
| - at variance : in disagreement, inconsistent | |
| - preside over : leading, beeing in charge | |
| - take steps to : take a decision to | |
| # | |
| - commitment : engagement | |
| - pragmatic : to be pratical, in action | |
| - mitigate : make something less severe, lessen the gravity | |
| - resilience : resistance, to recover quickly |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- pure-ftpd --- | |
| ## Change password | |
| $ sudo pure-pw passwd FTP_USER | |
| $ sudo pure-pw mkdb |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| # access tools to GCP | |
| - console | |
| - SDK | |
| - app mobile | |
| - REST API | |
| # filter | |
| Get only a value | |
| -> internal IP | |
| $ gcloud compute instances list --filter="name~'${NAME}' status=RUNNING" --format="value(networkInterfaces[0].networkIP)" | |
| -> public IP | |
| $ gcloud compute instances list --filter="name~'${NAME}' status=RUNNING" --format="value(networkInterfaces[0].accessConfigs[0].natIP)" | |
| In a formatted table (box) with sort | |
| $ gcloud compute instances list --format="table[box](name,creationTimestamp:sort=1,status,metadata.items.version,metadata.items.sha1,INTERNAL_IP)" --filter="name~'hylo' status=RUNNING" | |
| Get according to datetime (creationTimestamp) | |
| --> https://cloud.google.com/sdk/gcloud/reference/topic/datetimes | |
| $ gcloud *** list --filter="creationTimestamp<=-pXyYmZdAm" | |
| -p: beyond | |
| +p: from now | |
| # get startupscript log via instance serial port | |
| $ gcloud compute instances get-serial-port-output INSTANCE --zone=ZONE |awk '/GCEMetadataScripts/{print substr($0, index($0, $9))}' | |
| # USE-CASE | |
| + Compute | |
| - VM: Compute Engine | |
| - Auto-scaled VM: MIG | |
| - VMWare: VMWare Engine | |
| + Containers | |
| - app only: App Engine | |
| - container/image: Cloud Run | |
| - orchestrator: GKE | |
| - registry: Artifact Registry | |
| + Database | |
| - SQL: CloudSQL -> Cloud Spanner | |
| - NoSQL: Datastore -> BigTable (Time-series) | |
| - Live/real-time mobile/IoT: Firestore | |
| - App cache: Memorystore | |
| - Object: Cloud Storage | |
| - Disk: Persistent Disk -> Filestore | |
| + Network | |
| - Private network/ 1 project: VPC, Subnet | |
| - 1-for-all VPC: Shared VPC | |
| - Interco VPC: VPC Peering -> (mesh) NCC | |
| - Intern-only access to gAPI: PGA -> (ednpoint) PSC | |
| - Access to Tenant-VPC: PSA -> (PGA+PSA) PSC | |
| - Intern endpoint: PSC | |
| - Load balancer: Cloud LB | |
| - inter-cloud/on-prem: Cloud Internconnect | |
| - VPN: Cloud VPN | |
| - CDN: Cloud CDN | |
| - NAT: Cloud NAT | |
| - DNS: Cloud DNS | |
| - IP: External IP/ Private IP | |
| + Dev | |
| - CI/CD: Cloud Build | |
| - IaC: Deployment Manager | |
| - API endpoint: API Gateway | |
| - API management: Apigee | |
| - function only: Cloud Function | |
| - Message queue: Pub/Sub | |
| - Event-driven task: Eventarc | |
| + Security | |
| - Firewall (L3-4, VM) Firewall rules | |
| - WAF (L7, LB): Cloud Armor | |
| - Permimeter isolation: VPC-SC | |
| + IAM | |
| - IDaaS + Import Directory: Cloud Identity | |
| - RBAC + manage user: IAM + Service Account (GSA) | |
| - GKE IAM: Workload Identity (GSA + KSA) | |
| - Principle: At-least Privilege (3W) + Zero-Trust Access | |
| - Keys management: Cloud KMS | |
| + BigData/Analytics | |
| - Data warehouse: BigQuerry | |
| - Workflow orcherstrator: Cloud Composer (Apache Airflow) | |
| - Stream/batch process: DataFlow (Apache Beam) | |
| - Big data ecosystem: DataProc (Hadoop, Spark) | |
| - No-code/UI ETL: Data Fusion | |
| - BI platform: Looker | |
| + Observability | |
| - Log: Cloud Logging | |
| - Monitoring: Cloud Monitoring | |
| + AI | |
| - Generative AI: Vertex AI Studio + Gemini | |
| - MLOps: Vertex AI Platform | |
| - Media analyze: Vision AI | |
| - Text Translation: Cloud Natural Language & Translation | |
| --- WHY CLOUD | |
| # basic | |
| + cloud : rent a VM on internet --> rent a set of tools/services | |
| + need high performance | |
| - compute | |
| - storage | |
| + scaling | |
| scale-up (vert) --> scale-out (horiz) | |
| + budget | |
| /!\ reduce infra cost =! create value | |
| + cloud migration strategy | |
| - defensive : reduce cost | |
| - offensive : create value --> create new service | |
| + flexibility | |
| - rent compute : stop when wanted | |
| - try new services | |
| # 3 generation of cloud | |
| 1st: mutualize machines to secured space | |
| 2nd: (Amazon) move my infra to providers as API | |
| 3rd: serverless (rent infra as a service) --> infra adapt to apps | |
| # move to cloud reasons | |
| - create value from data/ business | |
| - build new cloud-native app/service | |
| - security | |
| - flexibility/pay what you try | |
| - cost optimization/ scaling | |
| --- HISTORY --- | |
| Basic principle: | |
| - collect data | |
| - stock data | |
| - process data with compute power | |
| Main type of compute power: | |
| - network | |
| - storage | |
| - compute engine | |
| ## Google technology timeline | |
| 2002: GFS (Filesystem/ OpenSource: HDFS) | |
| 2004: MapReduce (score Filesystem over datacenter) | |
| 2006: BigTable (NoSQL/ OpenSource: HBase) | |
| 2006: Borg (Container orchestrator/ OpenSource: Kubernetes) | |
| 2008: Dremel (Datawarehouse/ Origin of BigQuery) | |
| 2010: Colossus | |
| 2012: Flumejava | |
| 2013: Spanner | |
| 2015: Kubernetes | |
| 2016: Tensorflow | |
| ## GCP products categories | |
| + Large public, devices & productivity/collaboration/communication | |
| -> Gmail, Drive, Android | |
| + Legacy integrations (expose API, connect business platform) | |
| -> front app, apigee, vision/language API | |
| + Core apps (accelerate app delivery) | |
| -> app dev & management, AppEngine, ContainerEngine | |
| + Data in silos (EDW, Hadoop) | |
| -> Data analysis, ML, BigQuery, Dataflow | |
| + Private data center (Virtualization) (full managed operations) | |
| -> infrastructure, compute/container, storage, network | |
| --- GCP SERVICES | |
| # Compute | |
| - Compute Engine | |
| @ VM, migrate on-prem server, custom soft | |
| - GKE (Google Kubernetes Engine) | |
| container orcherstrator cluster | |
| @ manage containerized apps | |
| - Cloud Run | |
| IaaS container serverless platform, create your image/container, portable to k8s | |
| @ host containerized app, auto-scale based on traffic | |
| - (former) App Engine | |
| PaaS, focus on app runner only | |
| @ build/host web app without managing infra | |
| - Cloud Functions | |
| @ batch code runner | |
| - Artifact Registry | |
| @ image/container registry | |
| - Cloud VMWare Engine | |
| @ VMware workloads on GCP | |
| # Network | |
| - VPC (Virtual Private Cloud) | |
| private networking for cloud resources | |
| - Cloud LB | |
| distributes incoming traffic across multiple backend servers | |
| @ high availability | |
| - Cloud DNS | |
| DNS service (low latency/HA domain resolution) | |
| - Cloud NAT | |
| NAT gateway for private ressources | |
| - Cloud CDN | |
| caches content close to users around the world | |
| @ reduce latency | |
| - Cloud Interconnect | |
| connection between on-prem data center and Google Cloud | |
| - Cloud VPN | |
| - Network Intelligence Center | |
| network observability/monitoring | |
| - Cloud Router | |
| - External IP | |
| # Storage | |
| + Object storage | |
| - Cloud Storage | |
| large object storage | |
| @ unstructured data: backups, archives, multimedia files, static website content | |
| + Disk | |
| - Persistent disk | |
| block storage | |
| @ attach to VMs as hard drives | |
| - Filestore | |
| managed NAS (network-attached storage) | |
| @ traditional legacy apps, use NFS interface | |
| # Database | |
| + SQL/RDBMS | |
| - Cloud SQL | |
| relational database service (regional) | |
| @ MySQL, PostgreSQL, SQL Server | |
| - Cloud Spanner | |
| scalable relational/transactional db (global) | |
| @ massive, mission-critical app | |
| - AlloyDB | |
| PostgreSQL-compatible database | |
| @ top-tier transactional and analytical enterprise workloads | |
| + NoSQL | |
| (semi/unstructured data (doc, key-value pairs) -> real-time app, IoT, user profiles) | |
| - Firestore | |
| serverless NoSQL document database | |
| @ live, real-time syncing in mobile, web, IoT app | |
| - Cloud Bigtable | |
| high-perf NoSQL db (for ML) | |
| @ heavy analytical workloads and high-throughput data collection | |
| - Cloud Datastore | |
| NoSQL database | |
| - Memorystore | |
| in-memory data store Redis/Memcached | |
| @ sub-millisecond data access for caching and fast-retrieval app | |
| # Event process | |
| - Pub/Sub | |
| real-time/async messaging and event service | |
| @ connect app and streaming data | |
| - Eventarc | |
| event-driven task | |
| # Analytics/ BigData | |
| - Dataflow | |
| serverless Apache Beam service | |
| @ stream/batch processing | |
| - Dataproc | |
| Hadoop/Spark big data platform | |
| @ migrate existing big data ecosystems | |
| - BigQuery | |
| serverless, scalable enterprise data warehouse | |
| @ run blazing-fast SQL queries on massive datasets | |
| - Cloud Composer | |
| Apache Airflow service | |
| @ orchestrate workflows via Python code | |
| - Looker | |
| enterprise business intelligence (BI) platform | |
| @ create interactive dashboards, data app, embedded analytics | |
| - Cloud Data Fusion | |
| visual, no-code data integration tool (ETL) | |
| # AI/ML | |
| - Vertex AI | |
| unified, machine learning platform | |
| @ build, train, deploy custom ML models and generative AI app | |
| - Gemini Enterprise Agent Platform | |
| unified space for building AI agents | |
| @ utilize generative AI models securely in organization | |
| - Cloud Natural Language API | |
| analyzes text using deep learning models | |
| - Vision AI / Video Intelligence API | |
| pre-trained machine learning models | |
| @ analyze,classify images/videos | |
| - Speech-to-Text / Text-to-Speech | |
| AI-driven tools | |
| @ convert spoken audio into text | |
| - Cloud Machine Learning | |
| - Vision API | |
| - Speech API | |
| - Translate API | |
| - Cloud GPU (for ML) | |
| # IoT (Internet Of Thing) | |
| - Cloud IoT Core | |
| managing IoT devices | |
| # Identity/Security | |
| - IAM (Identity and Access Management) | |
| granular access control | |
| @ define who can take what action on specific cloud resources | |
| - Cloud KMS (Key Management Service) | |
| manage cryptographic keys used for cloud services | |
| - Google Cloud Armor | |
| DDoS protection and as WAF | |
| @ protect app from targeted threats | |
| - Firewall rules | |
| - Security Command Center | |
| comprehensive security management and data risk platform | |
| @ identify vulnerabilities/threats | |
| # Developer & Management Tools | |
| - Cloud Build | |
| continuous integration and continuous delivery (CI/CD) platform | |
| @ automates code deployment. | |
| - Deployment Manager | |
| IaC/CI-CD tools | |
| - API Gateway | |
| gateway for secure access to backend by REST API | |
| - Apigee | |
| native API management platform | |
| # Migration | |
| - RaMP (Rapid Migration and Modernization Program) | |
| migration assessment program | |
| - Migration Center | |
| on-prem/multi-cloud migration platform to GCP | |
| (old Stratozone) | |
| - Google Cloud Adoption Framework | |
| # Monitoring/Observability | |
| - Cloud Monitoring | |
| monitor performance metrics | |
| - Cloud Logging (Stackdriver) | |
| capture logs (centralized log management) | |
| # Command-Line tools | |
| - gcloud | |
| - gsutil | |
| - bq | |
| - kubectl | |
| --- INSTALL GCP SDK | |
| ## install | |
| https://cloud.google.com/sdk/docs/install#linux | |
| --- ENVIRONMENT VARIABLE | |
| Region: $REGION | |
| Project ID: $PROJECT_ID | |
| Project number: $PROJECT_NUMBER | |
| Zone: $ZONE | |
| Service Account: $SERVICE_ACCOUNT | |
| Service Account mail: $SERVICE_ACCOUNT_EMAIL | |
| --- PROJECT --- | |
| # usage | |
| + Global ressources collect/organization | |
| - track resource/quota usage | |
| - enable/disable service/API | |
| - control permissions/credentials | |
| - set billing account (can be associated with one/more projects) | |
| - any services is associated with one and only project | |
| + isolation between ressources | |
| - explicit trust across project can be created | |
| + specific security policies by project | |
| # GCP ressources hierarchy level | |
| --> https://cloud.google.com/blog/products/gcp/mapping-your-organization-with-the-Google-Cloud-Platform-resource-hierarchy | |
| ORG -> FOLDER -> PROJECT -> RESSOURCE | |
| [ORG] | |
| {Company} | |
| [FOLDERS] | |
| (Dept1) (Dept2) (SharedInfra) | |
| <TeamA> <TeamB> | |
| "Product1" "Product2" | |
| [PROJECTS] | |
| 'ProjectA' 'Project2' | |
| [RESSOURCES] | |
| -VM | |
| -storage | |
| # policy heritage | |
| - ressource inherit from parent | |
| - parent + ressource --> most permissive WIN | |
| # set default region/zone | |
| $ gcloud compute project-info add-metadata --metadata google-compute-default-region=REGION,google-compute-default-zone=ZONE | |
| --- IAM | |
| # principle | |
| + 3W = WHO can do WHAT in WHICH ressource | |
| + Least Privilege | |
| role with strict necessary permissions at the smallest scope (ressource, folder, project level) | |
| + Zero Trust access | |
| "never trust, always verify" | |
| based on user identity and context | |
| + service account | |
| account for apps for auth & interconnect with API | |
| + data encryption | |
| - data at rest: default encryption by Google-managed keys | |
| - data in transit: TLS | |
| # Users group | |
| - assign permissions to group than users | |
| - group own projet/ressources (PCA) | |
| - nested groups -> fine-grain control | |
| # Cloud Identity | |
| GCP Identity as a Service (IDaaS) | |
| @ syncs users/groups from on-prem Active Directory via Google Cloud Directory Sync (GCDS) | |
| # VPC Service Control (VPC-SC) | |
| service groupings as "Service Perimeters" where nothing enter/exist without your explicit permission | |
| @ prevent Reducing data exfiltration risks around resources | |
| # Cloud Key Management Service (KMS) | |
| centralized key management service | |
| + Customer-Managed Encryption Keys (CMEK) | |
| custom encryption keys for GCP services instead of Google's default keys | |
| # Identity-Aware Proxy (IAP) | |
| authent proxy service for VMs | |
| # RBAC (Role-Based Access Control) | |
| IAM (gcp RBAC) / Kubernetes RBAC | |
| + assign permissions to group not people | |
| - principal/identity : user, group, service-account | |
| - role : roles/viewer | |
| - binding (policy) : project, ressource, service, namespace | |
| # IAM roles type | |
| + primitive (project) | |
| Owner - Editor - Viewer | |
| + predefined | |
| for specific services (roles/cloudsql.editor) | |
| + custom | |
| user-defined for specific needs | |
| # List of all default Service Accounts | |
| - App Engine: PROJECT_ID@appspot.gserviceaccount.com | |
| - Compute Engine: PROJECT_NUMBER-compute@developer.gserviceaccount.com | |
| - Cloud Functions: PROJECT_ID@appspot.gserviceaccount.com | |
| - Cloud Run (fully managed): PROJECT_NUMBER-compute@developer.gserviceaccount.com | |
| - Cloud Build: PROJECT_NUMBER@cloudbuild.gserviceaccount.com | |
| - Google Kubernetes Engine (GKE): PROJECT_NUMBER-compute@developer.gserviceaccount.com | |
| - AI Platform: PROJECT_NUMBER@cloud-ml.google.com.iam.gserviceaccount.com | |
| - Dataprep: PROJECT_NUMBER@dataprep-rw.iam.gserviceaccount.com | |
| - Dataflow: PROJECT_NUMBER@dataflow-service-producer-prod.iam.gserviceaccount.com | |
| - BigQuery Data Transfer: PROJECT_ID@bigquerydatatransfer.iam.gserviceaccount.com | |
| - Pub/Sub: PROJECT_NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com | |
| # get role/permissions of a service account | |
| $ gcloud projects get-iam-policy PROJECT_ID --flatten="bindings[].members" --format='table(bindings.role)' --filter="bindings.members:SERVICE_ACCOUNT_EMAIL" | |
| $ gcloud asset search-all-iam-policies --query='policy:"serviceAccount:SERVICE_ACCOUNT_EMAIL"' | |
| # get Cloud Storage service agent account | |
| $ gcloud storage service-agent --project=PROJECT_ID | |
| or | |
| $ gsutil kms serviceaccount -p PROJECT_ID | |
| --- SECURITY | |
| # principle | |
| + Zero Trust | |
| + Least Priviledge | |
| # project ZERO | |
| detect all network breach (gcp, hyperscalers) | |
| # Audit Log | |
| - Cloud console activity page | |
| - stackdriver logging | |
| # Cloud Security Command Center | |
| - organization-level | |
| - global ressource dashboard | |
| - security, data risk platform | |
| - 3rd-part integration (DLP (Data Loss Prevention) API) | |
| # Forseti Security | |
| - vulnerability/breach analyzer | |
| - security auto enforcer | |
| --- APP ENGINE --- | |
| # type | |
| + App Engine Standard | |
| @ fast scale-to-zero in strict language sandboxes | |
| + App Engine Flexible | |
| uses Docker containers for custom runtimes and background tasks | |
| --- COMPUTE ENGINE --- | |
| # Preemptible (Spot) VMs | |
| discounted (80%) VMs - preempted at any time | |
| @ stateless/batch/fault-tolerant workloads (CI/CD worker) | |
| # Managed Instance Groups (MIGs) | |
| + Instance template | |
| + Auto-scaling | |
| adds/removes VMs in MIG based on metrics (CPU use, HTTP load balancing capacity) | |
| # Metadata | |
| + Startup/Shutdown script | |
| + ssh keys | |
| --- CLOUD FUNCTION --- | |
| serverless, single-purpose code snippets | |
| # Triggers (cloud events) | |
| - HTTP requests | |
| - Pub/Sub messages | |
| - Cloud Storage uploads | |
| --- DEVELOPER/IaC | |
| # custom image automation | |
| https://drive.google.com/file/d/1PdrkjwH0s3oRowU6sef6OAUEl_-W6mVi/view?usp=drive_link | |
| -base image |-> custom image -> startupscript/metadata |-> instance template | |
| -imported image |-> snapshot PD |-> cloud API | |
| # Deployment Manager | |
| # Cloud Marketplace | |
| - build by Google | |
| - 3rd-party vendors | |
| --- SECRET MANAGER --- | |
| ## show secret | |
| $ gcloud secrets versions access latest --secret SECRET_NAME --project PROJECT_NAME | |
| --- CLOUD STORAGE --- | |
| # networking/connectivity | |
| - not in VPC -> Google's production network - access by public endpoints | |
| - PGA (Private Google Access) allow internal-only services to connect to public google API (without going by internet) (like Google storage) | |
| # permissions | |
| + compute engine: | |
| - service account | |
| + GKE | |
| - Workload Identity | |
| # VM disk type | |
| - Persistent Disks (network-attached) | |
| - Local SSD (physically attached, ultra-high IOPS) | |
| # cloud Storage classes | |
| - Standard (frequent access) | |
| - Nearline (monthly access) | |
| - Coldline (quarterly access) | |
| - Archive (yearly access) | |
| *Object Lifecycle Management automates transitions classes | |
| # mount bucket to instance dir | |
| $ gcsfuse -o allow_other,nonempty --file-mode 755 --dir-mode 755 --gid XYZ --uid XYZ --key-file /PATH/TO/KEY.json BUCKET_NAME /MOUNT/POINT | |
| in fstab: | |
| BBUCKET_NAME /MOUNT/POINT gcsfuse rw,noauto,user,key_file=/PATH/TO/KEY.json | |
| unmount: | |
| $ fusermount -u /MOUNT/POINT | |
| or | |
| $ sudo umount /MOUNT/POINT | |
| --- DATABASE --- | |
| # Cloud SQL | |
| + High availability | |
| synchronous replication to a standby instance in another zone | |
| automated backups | |
| cross-region read-replicas | |
| + connection | |
| - private IP/ PSA (Private Service Access) | |
| - PSC (Private Service Connect) | |
| - google specific instance name/DNS | |
| - public IP: authorized networks | |
| - CloudSQL auth proxy | |
| - language specific connector lib (java, Python, Go) | |
| # Cloud Spanner | |
| + schema interleaving | |
| physically co-locates child table rows with parent table rows | |
| @ maximize read/write performance | |
| --- BIGDATA/ANALYTICS --- | |
| # Services | |
| + Storage & Data lake | |
| - GCS (Google Cloud Storage) | |
| bucket | |
| @ objects storage (raw, unstructured before processing) | |
| - Cloud Bigtable | |
| NoSQL db cluster | |
| @ massive analytical/operational data, very low latency, IoT | |
| + Data Warehouse & Analytics | |
| - BigQuery | |
| serverless datawarehouse enterprise | |
| @ run SQL query in massive data in seconds | |
| + Data Processing & ETL | |
| - Cloud Dataflow * only GCP/ automation | |
| streaming/batch data | |
| @ Apache Beam SDK | |
| - Cloud Dataproc * portable Spark Code - Run on VM/preemptible | |
| opensource big data frameworks | |
| @ Apache Hadoop/Spark/Hive/Presto, for on-prem migration | |
| - Cloud Dataprep * trifacta/ only batch | |
| visual dataset prep tool - no-code, for non-tech/ business analyst | |
| @ clean, explore, transform for analytics | |
| + Data Integration & Messaging | |
| - Cloud Pub/Sub | |
| queue/messaging | |
| @ event-driven | |
| - Cloud Data Fusion | |
| visual data integration pipeline + external source connection (API, SAP) | |
| @ CDAP framework, build no-code ETL pipelines | |
| + Orchestration & Management | |
| - Cloud Composer | |
| managed Apache Airflow - workflow orchestration automation | |
| @ set DAG (Directed Acyclic Graph) pipelines (python) | |
| - Dataplex | |
| unify data in storage/BigQuerry | |
| @ Data Fabric gouvernance/security | |
| # scenario | |
| + move existing on-prem Hadoop/Spark code? | |
| -> Dataproc | |
| + modern, SQL-based enterprise data warehouse? | |
| -> BigQuery | |
| + real-time streaming analytics (e.g., fraud detection)? | |
| -> Pub/Sub + Dataflow + BigQuery | |
| + cheap, raw file storage for a Data Lake? | |
| -> Cloud Storage (GCS) | |
| + low-latency storage for IoT device metrics (NoSQL)? | |
| -> Bigtable | |
| + stitch together 10 different data steps into one schedule? | |
| -> Cloud Composer | |
| + streaming clickstream data into BigQuery | |
| -> Pub/Sub - Dataflow - BigQuerry | |
| # BigQuerry | |
| + platform | |
| - compute (Dremel) | |
| - storage (Colossus) | |
| + External Tables | |
| @ query data in GCS (format Parquet/CSV) | |
| + Streaming insert | |
| @ load data into BigQuery row-by-row in real-time | |
| + Slots | |
| capacity-based pricing - pay flat fee for dedicated processing power | |
| VS pay-as-you-go / capped fee | |
| + SECURITY | |
| - Data Catalog policy tags | |
| @ Column-level security based on IAM | |
| # BigTable | |
| + schema design | |
| - revent 'hotspotting' when using timestamps as part of a row key | |
| @ hash/salt the row key -> writes distrubuted accross multiple nodes | |
| # Cloud Storage (GCS) - bucket | |
| + OLM (Object Lifecycle Management) | |
| @ manage storage class, object deletions | |
| # Composer | |
| + DAG (Directed Acyclic Graph) - Blueprint | |
| (Apache Airflow) tasks collection to run, organized by relationships/dependencies | |
| + architecture | |
| ------------------------ Orchestration (Cloud Composer) --------------------------------------- | |
| Data ingestion ---> Data lake/storage ---> Process ---> Datawarehouse/Analytics --> Visualization | |
| (dataset) - gcs + trigger DAG - DAG workflow - BigQuerry | |
| - API . dataproc | |
| - SQL db . ETL | |
| - App logs - BigQuerry SQL | |
| - Cloud function | |
| # Dataflow | |
| + Dataflow workers | |
| temp VM - use preemptible for finops | |
| + Windowing strategy | |
| continuous trigger input | |
| + PII Masking | |
| hide/transform sensitive data | |
| + ISSUE | |
| - handle event of late data (network latency) | |
| @ apache beam -> Watermarks (guess when all data in a certain window will arrived, ex: 10min) | |
| # Pub/Sub | |
| + processing timeline | |
| publisher (message) --> topics <--- (subscription) <-- (push/pull) subscibers | |
| + Dead-Learningetter topic | |
| handle failed-processed message (exceed max attempt) | |
| + in order delivery | |
| @ publisher add order keys to message | |
| + at-least-once delivery | |
| @ delivery garantee | |
| + Seek | |
| message retention acknowledge (up to 7d) | |
| --- NETWORK | |
| # network offer | |
| + Standard Tier | |
| GCP <-> PoP (ressource region) <-> Internet <-> ISP <-> user | |
| + Premium Tier | |
| GCP <-> Google Global Network <-> PoP <-> ISP <-> user | |
| # connectivity type | |
| + VPC (Virtual Private Cloud) | |
| - Accross global Google fiber network | |
| - VPC (global) -> subnet (regional) | |
| - Shared VPC | |
| 1 VPC in host project used by multiple projects (subnet, route, firewall) | |
| @ centralize IP/security management, secure inter-project communication | |
| -> Host Project (Central VPC) + X Service Project | |
| -> 1 org, X teams | |
| - VPC Peering | |
| internal interconnect two VPCs | |
| @ No Transitive Peering, No overlapping IP ranges | |
| -> X independent teams | |
| + PGA (Private Google Access) | |
| allow internal-only services to connect to public google API (without going by internet) (like Google storage) -> (set in VPC) | |
| -> Google's internal network intercepts traffic destined for Google APIs (like GCS) | |
| + PSC (Private Service Connect) | |
| private connections to a variety of services outside your VPC network | |
| connects consumers to services via a specific endpoint | |
| + PSA (Private Service Access) | |
| accessing specific Google services in your VPC network using private IP | |
| + Subnets | |
| Regional resources, define IP ranges per region | |
| # Hybrid Connectivity (On-Prem to Cloud) | |
| - Cloud VPN | |
| connect on-prem networks to GCP over internet (by IPsec) | |
| - Cloud Interconnect - Dedicated | |
| massive data transfer, strict private connection (no public internet) | |
| - Cloud Interconnect - Partner | |
| interconnect via an ISP | |
| - Direct Peering | |
| Connecting to G-Suite/YouTube (Public Google APIs), not for VPC access usually | |
| - HA VPN | |
| internet-based IPsec | |
| # Cloud LB | |
| + Global LB HTTPS | |
| - global anycast IP entry | |
| - cert manager | |
| + Content-base LB | |
| + Network LB | |
| TCP/UDP to regional compute | |
| + Internal LB | |
| regional/private TCP/UDP connection (VPN, on-prem) | |
| private IP | |
| # Network security | |
| + Firewall Rules: | |
| Applied to VM instances via Network Tags/Service Accounts | |
| allow/deny in/egress traffic via VM port | |
| + Cloud Armor | |
| Web Application Firewall (WAF) and DDoS protection | |
| To Load Balancer level. | |
| + Identity-Aware Proxy (IAP) | |
| Allows SSH/RDP into VMs without public IP (uses Google Identity authentication) | |
| + PGA (Private Google Access) | |
| allow internal-only services to connect to public google API (without going by internet) (like Google storage) - To set for VPC | |
| # Network Connectivity Center (NCC) | |
| managed service to automate routing between different networks | |
| Acts as"Hub", attach "Spokes" and handles the connectivity | |
| # Cloud DNS | |
| DNS service (low latency/HA domain resolution) @ PoP | |
| # Cloud NAT | |
| NAT service/gateway for outbound traffic to internet | |
| @ assign public IP to private ressouces | |
| # Cloud CDN | |
| # GKE network | |
| + VPC-native: GKE insite VPC (project wide), pods IPs routable in VPC | |
| - Nodes range | |
| - Pods range | |
| - Services range | |
| + internal cluster networking (Pod-to-Pod) - CNI (Container Network Interface) | |
| - GKE Dataplane V2 | |
| /!\ - (old) Kubenet | |
| + cross-PROJECT connection | |
| - VPC Peering | |
| - connectivity permission -> Workload Identity (map GKE Service Account to GCP Service Account) | |
| + Egress (external connection) | |
| - cloud NAT gateway (Pod -> Node -> Cloud NAT -> internet) | |
| /!\ - public IP to internet (Node -> Internet) | |
| + Ingress | |
| - GCP Load Balancing (L4, TCP/UDP traffic, direct routing) | |
| - Ingress Controller (Application Load Balancer (L7, HTTP/HTTPS, APIs)) | |
| - API Gateway (micros-services, mesh) | |
| + to On-Prem/External cloud | |
| (hybrid connectivity) | |
| - Cloud VPN | |
| - Cloud Interconnect | |
| # Location | |
| + Multi-Region (Continent) | |
| - US | |
| - EU | |
| - ASIA | |
| + Region (Country) | |
| + Zone (Town) | |
| | Location | Resources | |
| +---------------+------------------------------------------------------------------------- | |
| | Global | VPC, Firewalls, Routes, VM Images, Global LB | |
| | Multi-Region | GCS, BigQuery, Cloud Spanner, Datastore, App Engine | |
| | Region | Subnets, Regional PD, Static External IP | |
| | Zone | VMs, Zonal PD, Machine Types | |
| # IN/OUT VPC network | |
| + VPC | |
| Compute Engine, GKE | |
| @ IP | |
| + Core Prod Network | |
| - Cloud Storage, BigQuery, Pub/Sub, Cloud Run, Cloud Functions, App Engine | |
| - gAPI (Vision API, Translation API) | |
| @ public IP/endpoint (*.googleapis.com) + PGA | |
| + Tenant/Service Producer VPC | |
| Cloud SQL, Memorystore, Filestore, Vertex AI | |
| @ VPC peering + PSA/PSC | |
| --- LOAD BALANCER --- | |
| + type | |
| - L7 (HTTP/S): Application LB | |
| Understads HTTP headers, cookies, URLs. | |
| Path-based routing (/video, /images). | |
| - L4 (TCP/UDP): Network LB | |
| forwards packets based on IP/Port. Faster | |
| + scope | |
| - Global | |
| traffic across regions. Single Anycast IP address worldwide | |
| (HTTPs Load Balancer, SSL Proxy, TCP Proxy) | |
| - Regional | |
| traffic stays in one region | |
| (Network Load Balancer) | |
| + access | |
| - Internal: Only from private RFC1918 IPs (inside VPC) | |
| - External: from internet | |
| --- GOOGLE KUBERNETES ENGINE (GKE) | |
| # principle | |
| + high available, multi-zonal cluster | |
| Deploy "Regional" cluster | |
| (auto replicates control plane/worker nodes across multiple zones for high availability) | |
| # Google Cloud Anthos | |
| unified multicloud platform | |
| @ manage Kubernetes clusters across GCP, on-prem, other cloud | |
| --- MIGRATION | |
| # RaMP process - Project management | |
| + Assess | |
| + Plan | |
| + Migrate | |
| + Innovate | |
| # Google Cloud Adoption Framework | |
| assessment methodology for cloud readiness by: | |
| Learn, Lead, Scale, Secure | |
| --- MONITORING/OBSERVABILITY | |
| # Cloud logging sinks | |
| rules to route logs to external destinations | |
| (Cloud Storage (archival), BigQuery (analytics), Pub/Sub (streaming)) | |
| # APM (Application Performance Monitoring) | |
| - Cloud Trace | |
| tracks request latency across microservices | |
| - Cloud Profiler | |
| continuous analyzes CPU/memory consumption in prod | |
| @ find code bottlenecks | |
| # SRE (Site Reliability Engineer) concepts/kpi | |
| - SLI (Service Level Indicator) - Measure | |
| metric to measure level of service provided | |
| EX: percentage of successful HTTP 200 | |
| - SLO (Service Level Objective) - Goal | |
| target value/range of values for an SLI | |
| EX: goal of 99.9% success rate for the endpoint | |
| - SLA (Service Level Agreement) - Contract | |
| contract agreement with users (penalties if targets missed) | |
| EX: if service falls below 99.5% success | |
| - Error Budget - Buffer | |
| amount of "unreliability" permitted before releasing new features is paused | |
| EX: if SLO is 99.9%, error budget is 0.1% | |
| with 1,000,000 requests/month, "allowed" to have 1,000 failed requests | |
| SRE Rule: If budget used (major outage consumes 800 failures), | |
| stop pushing new features and focus on system reliability until the budget recovers | |
| # DRP (Disaster Recovery Plan) | |
| - RPO (Recovery Point Objective) - Data Loss | |
| max tolerable period of data loss measured | |
| EX: can afford 15min to lose data before restore backup | |
| - RTO (Recovery Time Objective) - Downtime | |
| max tolerable duration of time a system can be down | |
| EX: can be offline for 1h before failover/recovery process | |
| # FinOPS | |
| + cost-optimized workloads | |
| - serverless: Cloud Run (scale to zero if not needed) | |
| - preemptible/Spot VMs | |
| - strict billing budget alerts | |
| # Regulatory compliance | |
| + Healthcare compliance (HIPAA, GDPR) | |
| - Cloud Healthcare API | |
| - VPC Service Controls enabled | |
| - CMEK encryption enforced | |
| - Cloud Audit Logs mandated | |
| - Google signed Business Associate Agreement (BAA) | |
| --- CERTIFICATION | |
| # GCP Cloud Digital Leader | |
| + 4 area keys | |
| - ability of GCP to digitally transform the organization | |
| - services available on GCP | |
| - tools available on GCP | |
| - AI & machine learning |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- DOC --- | |
| # Git branching strategy | |
| + GitFlow | |
| Master - Develop - Feature - Release - Hotfix | |
| + GitHub Flow | |
| Main <- PR | |
| + Gitlab Flow | |
| Main <- PR -> DEV-ENV | |
| -> PROD-ENV | |
| + TBD (Trunk-Based Development) | |
| Main/Trunk <- Feature-2 | |
| <- Feature-1 | |
| --- MISC --- | |
| ## execute dry-run command | |
| $ git cmd --dry-run | |
| ## remove tracked dir/file on remote but keep on local | |
| $ git rm --cached <file-name> | |
| $ git rm -r --cached <folder-name>git checkout -b subbranch_of_b1 branch1 | |
| ## prevent git from detecting changes in some files | |
| $ git update-index --assume-unchanged | |
| ## get branch name via cmd | |
| $ LONG_SHA=$(git rev-parse HEAD) | |
| $ BRANCH_NAME=$(sed -nE "/$LONG_SHA/s/.*refs\/heads\/(.*)/\1/p" <<< "$(git ls-remote -q)") | |
| ## set remote URL | |
| SSH | |
| $ git remote set-url origin git@DOMAIN:PROJECT/REPOSITORY.git | |
| HTTPS | |
| $ git remote set-url origin https://DOMAIN/PROJECT/REPOSITORY.git | |
| ## set pre-commit | |
| -> https://github.com/pre-commit/pre-commit | |
| ## BISECT: debug commit by commit | |
| $ git bisect start | |
| if bad | |
| $ git bisect bad | |
| if good | |
| $ git bisect good | |
| $ git bisect run | |
| $ git bisect log | |
| $ git bisect skip | |
| --- ISSUE | |
| ## Gitlab Token changed, reload local repo with new token | |
| "remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://gitlabee.dt.renault.com/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied | |
| fatal: Authentication failed for..." | |
| $ git remote set-url origin https://USER:GITLAB_TOKEN@GITLAB_URL/REPO.git | |
| --- CONFIG --- | |
| ## enable color | |
| $ git config --global color.ui true | |
| --- COMMIT --- | |
| ## verify and select commits | |
| $ git add -p | |
| ## undo git add | |
| $ git reset | |
| ## ammend to the previous commit | |
| Commit without edit message | |
| $ git commit --amend --no-edit | |
| Change commit message in previous commits | |
| $ git rebase -i HEAD~n | |
| --> reword SHA1 | |
| $ git commit --ammend -m "NEW MESSAGE" | |
| ## change author of commit | |
| Set the correct author for current Git repo then | |
| $ git rebase SRC_BRANCH --exec "git commit --amend --reset-author --no-edit" | |
| or | |
| $ git rebase --onto HEAD~N --exec "git commit --amend --reset-author --no-edit" HEAD~N | |
| ## revert file to a state from other branch | |
| $ git checkout BRANCH -- PATH/TO/FILE | |
| ## get long sha1 of head | |
| $ git rev-parse HEAD | |
| ## look for branches containing a commit | |
| $ git branch (--remote) --contains COMMIT_SHA | |
| ## commit format | |
| + Default format | |
| TYPE(SCOPE): SUBJECT | |
| OPTIONAL BODY | |
| OPTIONAL FOOTER | |
| + Merge commit | |
| Merge branch 'BRANCH_NAME' | |
| + Type | |
| - feat: adds a new feature | |
| - fix: fixes a bug | |
| - refactor: rewrite/restructure code, however does not change any behaviour | |
| - perf: special refactor commits, that improve performance | |
| - style: do not affect the meaning (white-space, formatting, missing semi-colons, etc) | |
| - test: add missing tests or correcting existing tests | |
| - docs: affect documentation only | |
| - build: affect build components like build tool, ci pipeline, dependencies, project version, ... | |
| - ops: affect operational components like infrastructure, deployment, backup, recovery, ... | |
| - chore: miscellaneous commits e.g. modifying .gitignore | |
| + Breaking Changes Indicator "!" | |
| TYPE(SCOPE)!: SUBJECT | |
| --- TAG --- | |
| $ git tag TAG BRANCH | |
| $ git push origin TAG | |
| ## merge tag to branch | |
| $ git checkout BRANCH | |
| $ git merge vTAG | |
| --- FIXUP --- | |
| ## fixup anterior commit | |
| $ git commit --fixup=SHA1 | |
| $ git rebase -i --autosquash SHA1~1 | |
| --- BRANCH --- | |
| ## create branch from another branch | |
| $ git checkout branch_orig | |
| $ git checkout -b feature/branch_dest branch_orig | |
| ## diff/compare branches | |
| $ git diff BRANCHE1..BRANCHE2 | |
| ## rename branch | |
| $ git branch -m NEW_NAME | |
| $ git push origin :OLD_NAME NEW_NAME | |
| $ git push origin -u NEW_NAME | |
| --- REBASE --- | |
| ## rebase feat branch from an origin branch | |
| $ git rebase -i master | |
| --- RESTORE --- | |
| ## restore file to a commit | |
| $ git checkout COMMID_ID -- FILE | |
| ## restore file to a branch | |
| $ git restore --source origin/BRANCH FILE | |
| --- CHERRY-PICK --- | |
| ## cherry pick commit from other branch | |
| $ git checkout TARGET_BRANCH | |
| $ git cherry-pick SHA1 | |
| --- HISTORY | |
| # blame: identify who make a change on a file | |
| $ git blame FILE | |
| # log | |
| + log of single file | |
| $ git glog --follow -p PATH/TO/FILE | |
| + log overiew of all commits and changes | |
| $ git log --patch | |
| # REFLOG (reference logger) log all commands | |
| $ git reflog show BRANCH | |
| --- MERGE --- | |
| ## manual merge local/origin branch to master | |
| Fetch and check out the branch (create it on local if not exist) | |
| $ git fetch origin | |
| $ git checkout -b "feat/MY_BRNACH" "origin/feat/MY_BRANCH" | |
| Review the changes locally | |
| Merge the branch and fix any conflicts that come up | |
| $ git fetch origin | |
| $ git checkout "master" | |
| $ git merge --no-ff "feat/MY_BRANCH" | |
| Push | |
| $ git push origin "master" | |
| --- RESET --- | |
| ## reset to origin branch | |
| $ git reset --hard origin/BRANCH | |
| --- STASH ---- | |
| ## stash workflow | |
| Save | |
| $ git stash save | |
| or | |
| git stash save "WIP MESSAGE" | |
| Include untracked files | |
| $ git stash save --include-untracked | |
| List | |
| $ git stash list | |
| Show desc | |
| $ git stash show stash@{0} | |
| Get stashed work (and delete ref) | |
| $ git stash pop | |
| or | |
| $ git stash pop stash@{0} | |
| Get but don't delete ref | |
| $ git stash apply | |
| --- CLEAN --- | |
| ## clean repo directory, remove untracked files | |
| $ git clean | |
| -n: dry-run | |
| -f: force | |
| -i: interactive | |
| -d: directory+files | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # github best-practices | |
| + set default branch | |
| + branch protection | |
| - branch ruleset | |
| status -> user bypass -> target branch -> rules -> restrictions | |
| - classic protection rule |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ------ GITLAB-CI ------ | |
| ## scheduled job | |
| --- | |
| JOB:on-schedule: | |
| stage: STAGE | |
| variables: | |
| VARS: VALUE | |
| script: | |
| - CMD | |
| only: | |
| - schedules | |
| --- | |
| ## templating bloc | |
| --- | |
| .TEMPLATE_NAME | |
| bloc: | |
| key: value | |
| JOB: | |
| extend: | |
| - .TEMPLATE_NAME | |
| --- | |
| ## loop/ parallel matrix | |
| .action: &action | |
| image: IMAGE | |
| script: | |
| - command | |
| needs: [] | |
| when: manual | |
| job-on-ENV: | |
| stage: ENV | |
| <<: *action | |
| parallel: | |
| matrix: | |
| - ENV: [APP-1, ENV-2 , ENV-3] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- chart library | |
| ## use library | |
| + in Chart template | |
| --> Chart.yaml | |
| dependencies: | |
| - name: my-library | |
| version: 1.0.0 | |
| repository: "file://PATH/TO/LIBRARY" | "https://REPO.LINK" | |
| $ helm dependency update | |
| |____charts | |
| |____chart-lib-1.0.0.tgz | |
| # directory structure | |
| project/ | |
| ├── .github/workflows/ | |
| ├── charts/ | |
| │ ├── app1/ | |
| │ │ ├── Chart.yaml | |
| │ │ ├── values.yaml # Default fallback values | |
| │ │ └── templates/ | |
| │ ├── deployment.yaml | |
| │ ├── service.yaml | |
| │ ├── ingress.yaml | |
| │ ├── pdb.yaml | |
| │ ├── hpa.yaml | |
| │ └── secrets.yaml | |
| │ └── app2/ | |
| │ | |
| └── environments/ | |
| ├── dev/ | |
| │ ├── app1-values.yaml # override values | |
| │ └── database-values.yaml | |
| └── prod/ | |
| ├── app1-values.yaml | |
| └── database-values.yaml | |
| # instaml with helm (+merge values) | |
| $ helm upgrade --install APP ./charts/app1 \ | |
| -f ./charts/app1/values.yaml \ | |
| -f ./environments/production/app1.yaml \ | |
| --namespace production | |
| --- UMBRELLA | |
| template used by other charts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## 1xx INFORMATIONAL | |
| 100 - Continue - Client should continue with request. | |
| 101 - Switching Protocols - Server is switching protocols. | |
| 102 - Processing - Server has received and is processing the request. | |
| 103 - Processing - Server has received and is processing the request. | |
| 122 - Request-uri too long - URI is longer than a maximum of 2083 characters. | |
| ## 2xx SUCCESS | |
| 200 - Ok - The request was fulfilled. | |
| 201 - Created - Following a POST command, this indicates success, but the textual part of the response line indicates the URI by which the newly created document should be known. | |
| 202 - Accepted - The request has been accepted for processing, but the processing has not been completed. The request may or may not eventually be acted upon, as it may be disallowed when processing actually takes place. there is no facility for status returns from asynchronous operations such as this. | |
| 203 - Partial Information - When received in the response to a GET command, this indicates that the returned metainformation is not a definitive set of the object from a server with a copy of the object, but is from a private overlaid web. This may include annotation information about the object, for example. | |
| 204 - No Response - Server has received the request but there is no information to send back, and the client should stay in the same document view. This is mainly to allow input for scripts without changing the document at the same time. | |
| 205 - Reset Content - Request processed, no content returned, reset document view. | |
| 206 - Partial Content - partial resource return due to request header. | |
| 207 - Multi-Status - XML, can contain multiple separate responses. | |
| 208 - Already Reported - results previously returned. | |
| 226 - Im Used - request fulfilled, reponse is instance-manipulations. | |
| ## 3xx REDIRECTION | |
| 301 - Moved - The data requested has been assigned a new URI, the change is permanent. (N.B. this is an optimisation, which must, pragmatically, be included in this definition. Browsers with link editing capabiliy should automatically relink to the new reference, where possible) | |
| 302 - Found - The data requested actually resides under a different URL, however, the redirection may be altered on occasion (when making links to these kinds of document, the browser should default to using the Udi of the redirection document, but have the option of linking to the final document) as for "Forward". | |
| 303 - Method - Like the found response, this suggests that the client go try another network address. In this case, a different method may be used too, rather than GET. | |
| 304 - Not Modified - If the client has done a conditional GET and access is allowed, but the document has not been modified since the date and time specified in If-Modified-Since field, the server responds with a 304 status code and does not send the document body to the client. | |
| 305 - Use Proxy - Content located elsewhere, retrieve from there. | |
| 306 - Switch Proxy - Subsequent requests should use the specified proxy. | |
| 307 - Temporary Redirect - Connect again to different URI as provided. | |
| 308 - Permanent Redirect - Connect again to a different URI using the same method. | |
| ## 4xx CLIENT SIDE ERRORS | |
| 400 - Bad Request - The request had bad syntax or was inherently impossible to be satisfied. | |
| 401 - Unauthorized - The parameter to this message gives a specification of authorization schemes which are acceptable. The client should retry the request with a suitable Authorization header. | |
| 402 - Payment Required - The parameter to this message gives a specification of charging schemes acceptable. The client may retry the request with a suitable ChargeTo header. | |
| 403 - Forbidden - The request is for something forbidden. Authorization will not help. | |
| 404 - Not Found - The server has not found anything matching the URI given. | |
| 405 - Method Not Allowed - Request method not supported by that resource. | |
| 406 - Not Acceptable - Content not acceptable according to the Accept headers. | |
| 407 - Proxy Authentication Required - Client must first authenticate itself with the proxy. | |
| 408 - Request Timeout - Server timed out waiting for the request. | |
| 409 - Conflict - Request could not be processed because of conflict. | |
| 410 - Gone - Resource is no longer available and will not be available again. | |
| 411 - Length Required - Request did not specify the length of its content. | |
| 412 - Precondition Failed - Server does not meet request preconditions. | |
| 413 - Request Entity Too Large - Request is larger than the server is willing or able to process. | |
| 414 - Request URI Too Large - URI provided was too long for the server to process. | |
| 415 - Unsupported Media Type - Server does not support media type. | |
| 416 - Requested Rage Not Satisfiable - Client has asked for unprovidable portion of the file. | |
| 417 - Expectation Failed - Server cannot meet requirements of Expect request-header field. | |
| 418 - I'm a teapot - I'm a teapot. | |
| 420 - Enhance Your Calm - Twitter rate limiting. | |
| 421 - Misdirected Request - Server is not able to produce a response. | |
| 422 - Unprocessable Entity - Request unable to be followed due to semantic errors. | |
| 423 - Locked - Resource that is being accessed is locked. | |
| 424 - Failed Dependency - Request failed due to failure of a previous request. | |
| 426 - Upgrade Required - Client should switch to a different protocol. | |
| 428 - Precondition Required - Origin server requires the request to be conditional. | |
| 429 - Too Many Requests - User has sent too many requests in a given amount of time. | |
| 431 - Request Header Fields Too Large - Server is unwilling to process the request. | |
| 444 - No Response - Server returns no information and closes the connection. | |
| 449 - Retry With - Request should be retried after performing action. | |
| 450 - Blocked By Windows Parental Controls - Windows Parental Controls blocking access to webpage. | |
| 451 - Wrong Exchange Server - The server cannot reach the client's mailbox. | |
| 499 - Client Closed Request - Connection closed by client while HTTP server is processing. | |
| ## 5xx SERVER SIDE ERROR | |
| 500 - Internal Error - The server encountered an unexpected condition which prevented it from fulfilling the request. | |
| 501 - Not Implemented - The server does not support the facility required. | |
| 502 - Service temporarily overloaded - The server cannot process the request due to a high load (whether HTTP servicing or other requests). The implication is that this is a temporary condition which maybe alleviated at other times. | |
| 503 - Gateway timeout - This is equivalent to Internal Error 500, but in the case of a server which is in turn accessing some other service, this indicates that the respose from the other service did not return within a time that the gateway was prepared to wait. As from the point of view of the clientand the HTTP transaction the other service is hidden within the server, this maybe treated identically to Internal error 500, but has more diagnostic value. | |
| 504 - Gateway Timeout - Gateway did not receive response from upstream server. | |
| 505 - Http Version Not Supported - Server does not support the HTTP protocol version. | |
| 506 - Variant Also Negotiates - Content negotiation for the request results in a circular reference. | |
| 507 - Insufficient Storage - Server is unable to store the representation. | |
| 508 - Loop Detected - Server detected an infinite loop while processing the request. | |
| 509 - Bandwidth Limit Exceeded - Bandwidth limit exceeded. | |
| 510 - Not Extended - Further extensions to the request are required. | |
| 511 - Network Authentication Required - Client needs to authenticate to gain network access. | |
| 598 - Network Read Timeout Error - Network read timeout behind the proxy. | |
| 599 - Network Connect Timeout Error - Network connect timeout behind the proxy. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| # why IaC | |
| - DRP: repeat, redeploy infra, backup | |
| - PCA: doc maintainance infra, traceability, human readable | |
| - version control |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- PRESENTATION | |
| Je suis Lead DevOps & Cloud Architect avec 9 ans d'experience. | |
| Je suis certifie et specialise dans les environnements Linux, l'automatisation CI/CD, | |
| l'Infrastructure as Code (IaC) et les architectures Cloud, | |
| principalement sur GCP, mais aussi AWS, Azure et Private Cloud. | |
| Mon parcours m'a amene a piloter des projets complexes d'architecture et de migration cloud | |
| (ex. Groupe Rocher, Renault Groupe, AGIRC ARRCO), | |
| a manager/guider des equipes techniques en environnement Agile (Scrum, SAFe), | |
| et plus recemment a integrer des briques d'I.A (GenAI, Vertex AI, RAG) | |
| dans les operations DevOps et l'observabilite | |
| En parallele, je suis formateur et speaker lors d'evenements tech (DevFest, | |
| Google Next/IO, CNCF/KCD) | |
| --- SPEECH | |
| # STAR method | |
| + S - Situation: Set the scene | |
| Briefly describe the context | |
| + T - Task: Define the challenge | |
| Explain the problem to solve, the goal you needed to achieve | |
| + A - Action: Describe what YOU did | |
| Detail the specific steps, focus on your individual contribution by using "I" instead of "We" | |
| + R - Result: Share the outcome | |
| Explain the positive impact of your actions, using quantifiable metrics (numbers, percentages, time saved) | |
| # intro | |
| # ending | |
| - Thanks for this interview, | |
| if you want more informations, don't hesitate to contact me, | |
| and if you want us to meet in person, it will be a pleasure to come | |
| --- MISC | |
| # dev benefits from DevOps | |
| - automation | |
| - autonomy | |
| - light process | |
| - quick review/test | |
| - environment availability | |
| # prod major outage/incident | |
| + problems | |
| + solution | |
| # bad hierarchy decisions | |
| + why | |
| + challenge | |
| # ArgCD application sets | |
| code application template | |
| --- KUBERNETES | |
| # operator | |
| + type of operator | |
| - database | |
| - cert-manager | |
| - external-secret | |
| - monitoring-agent | |
| --- QUESTIONS/ANSWERS | |
| # how do you manage arbitrary decision | |
| - what you undergo is not your fault, what you do with is your responsability |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --> https://jinja.palletsprojects.com/en/3.0.x/templates/#jinja-filters.indent | |
| ## condition | |
| {% if VAR == VALUE %} or {% if VAR is defined %} | |
| PUT THIS | |
| {% endif %} | |
| ## indent multiline var | |
| vars: | |
| level_1: | |
| level_2: | | |
| - list_1 | |
| - list_2 | |
| --- | |
| main: | |
| - this_is_a_list: | |
| {{ level_1.level_2 | indent( width=4, first=False) }} | |
| --- | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC | |
| # features | |
| - auto rollout/rollback | |
| - auto-healing | |
| - service discovery | |
| --- | |
| # Architecture | |
| + Control Plane (Master Node) | |
| - API Server | |
| Everything (kubectl, nodes, scheduler) talks only to the API Server. | |
| Validates requests | |
| - etcd: | |
| The memory, Key-value store | |
| Holds the entire state of the cluster. | |
| - Scheduler | |
| The matchmaker | |
| Decides where to put a new Pod based on resources (CPU/RAM) and constraints (Taints/Tolerations). | |
| - Controller Manager | |
| watches the state (e.g., "I need 3 replicas") and ensures the current state matches it | |
| (e.g., "Oh, one died? Start a new one"). | |
| + Data Plane (Worker Nodes) | |
| Where apps run. | |
| - Kubelet | |
| talks to the API server | |
| ensures containers are running in the Pod | |
| - Kube-proxy | |
| maintains network rules (IP tables/IPVS) on nodes | |
| allow communication between Pods and Services | |
| - Container Runtime | |
| (containerd, CRI-O, Docker Engine) runs the containers | |
| # Core Objects | |
| + Deployment | |
| Manages Pods | |
| creates ReplicaSet, which spins up N Pods (ensure RollingUpdate) | |
| + ReplicaSets (formerly ReplicationController) | |
| Group of Pod replicas, | |
| create Pods | |
| + Pod | |
| server who run the container (smallest deployable unit), stateless | |
| 1 container per pod, or 2 (Sidecars) | |
| + StatefulSet | |
| for stateful applications (databases) | |
| with persistent identifier & dedicated persistent Volume | |
| + DaemonSet | |
| specific Pod that runs on every Node in the cluster | |
| (for monitoring agent) | |
| + Job | |
| batch processing | |
| runs a Pod to completion | |
| + CronJob | |
| runs Jobs on schedule | |
| + Service | |
| networking endpoint. | |
| Gives a static IP/DNS name to a set of Pods. | |
| - ClusterIP (Internal only) | |
| - NodePort (Open port on VM) | |
| - LoadBalancer (Cloud LB) | |
| + Ingress | |
| HTTP/HTTPS router | |
| in front of Services to handle SSL termination | |
| path-based routing (/api, /web) and host routing (app.example.com). | |
| + ConfigMap & Secret | |
| configuration from the container image | |
| key-value data (env vars, config files) | |
| # Storage/Volume (Stateful vs. Stateless) | |
| + PVC (Persistent Volume Claim) | |
| requested storage by a Pod | |
| + PV (Persistent Volume) | |
| disk, storage created | |
| + StorageClass | |
| type of storage (Fast SSD, Cheap HDD). | |
| Allows dynamic provisioning | |
| + StatefulSet | |
| pod for stateful application (databases: Cassandra, Kafka, Postgres) | |
| stable Network IDs (web-0, web-1) | |
| with Persistent Storage (Disk sticks to web-0 even if it restarts) | |
| # Scheduling & Resource Management | |
| + Requests & Limits | |
| - Requests | |
| Guaranteed resources | |
| - Limits | |
| hard threshold (RAM limit -> OOMKilled, CPU limit -> Throttled/slow) | |
| + Taints & Tolerations | |
| - Taint | |
| Applied to a Node ("No one enter unless you have a ticket"). | |
| - Toleration | |
| Applied to a Pod ("I have the ticket"). | |
| + Affinity / Anti-Affinity | |
| - Affinity: "I want to run on a node with SSDs." | |
| - Anti-Affinity/ topologySpreadConstraints: "Do not run two copies of this App on the same node" (High Availability). | |
| # Security & Organization | |
| + Namespace | |
| virtual separation in cluster | |
| + ServiceAccount | |
| identity for processes that run in a Pod | |
| + Role & ClusterRole | |
| set of permissions for cluster | |
| + RoleBinding & ClusterRoleBinding | |
| Grants the permissions defined in a Role to a user or a ServiceAccount | |
| # Deployement strategy (HA deployment) | |
| + rolling update | |
| - maxUnavailable (set 0) | |
| - maxSurge (25% / 1) | |
| - readinessProbe (endpoint /health) | |
| - gracefull shutdown (terminationGracePeriodSeconds 30s++) | |
| - Pod Disruption Budgets (PDB) (minAvailable: 2) | |
| - podAntiAffinity | |
| --- NETWORK | |
| # core networking | |
| + Pod-2-Pod (no NAT) | |
| + Agent to all pods (DaemonSet/ Kubelet) | |
| --> installed on a node | |
| + "IP-per-Pod" Model (like VM) | |
| --> IP consistency | |
| # traffic flows | |
| + container-2-container : by localhost | |
| + pod-2-pod SAME-NODE : by node internal ethernet bridge (cbr0, docker0) | |
| + pod-2-pod CROSS-NODE : by CNI (Container Network Interface) | |
| # Service Types | |
| + ClusterIP (Default) | |
| cluster-internal IP, reachable only within the cluster | |
| + NodePort | |
| expose by specific port (30000-32767) on Node's IP | |
| + LoadBalancer | |
| cloud provider's LB for provision public IP to routes to NodePorts. | |
| + ExternalName | |
| DNS alias (CNAME), redirect traffic to external DNS | |
| --- PROPERTIES | |
| # pod affinity | |
| place pod on same node | |
| make network communication lightning fast. | |
| + Usage | |
| - Web Server & Local Cache | |
| web frontend needs to fetch data from an in-memory cache extremely fast | |
| - Data Processing | |
| heavy data-processing between 2 apps | |
| + topologyKey | |
| - Node level | |
| - Zone level | |
| + Affinity Family | |
| - Node Affinity | |
| - Pod Affinity | |
| - Pod Anti-Affinity | |
| --- TOOLS | |
| # cluster admin | |
| + kOps | |
| setup k8s cluster (instead of kubeadmin) | |
| # Package Management & Configuration | |
| + Helm: | |
| package manager to deploy applications | |
| use configured templates (charts), only values to fill | |
| + Kustomize | |
| patch manifest for different envs | |
| use template YAML | |
| + Ytt (Carvel) | |
| structure-aware YAML templating tool | |
| # Infrastructure as Code (IaC) | |
| + Terraform / OpenTofu | |
| manage k8s resources (namespaces, deployments, configmaps) using HCL | |
| + Pulumi | |
| deploy cloud infrastructure and k8s resources | |
| use real programming languages (TypeScript, Python, Go). | |
| # CI/CD & GitOps | |
| + Argo CD | |
| automates application deployment (GITOPS) | |
| continuously syncing cluster's state with Git repo | |
| + Flux | |
| continuous monitors Git repo and applies config changes/image updates to cluster | |
| + Spinnaker | |
| multi-cloud continuous delivery platform | |
| manage complex, orchestrated rollouts (canary, blue/green) | |
| + Tekton | |
| framework to create standardized, cloud-native CI/CD pipelines that run directly within k8s | |
| # Monitoring & Observability | |
| + Prometheus | |
| Collects real-time metrics data | |
| monitor cluster/applicationsthe health and performance | |
| + Grafana | |
| visual dashboards for collected metrics data | |
| + Jaeger | |
| traces user requests across different microservices | |
| help to identify performance bottlenecks and bugs | |
| # Service Mesh | |
| + Istio | |
| communication between microservices without requiring you to change their underlying code. | |
| + Linkerd | |
| service mesh for securing service-to-service traffic | |
| + Envoy | |
| proxy service for cloud-native apps | |
| # Ingress & API Gateways | |
| + NGINX Ingress Controller | |
| manages external access (route HTTP/S traffic) | |
| reverse proxy and load balancer. | |
| - INGRESS-NGINX (k8s community EOL 2026) | |
| - NGINX-INGRESS (F5 / NGINX Inc) | |
| + HAProxy | |
| + Envoy Gateway | |
| + NGINX Gateway Fabric (NGF) | |
| + Istio Gateway | |
| --> Gateway API | |
| + Traefik | |
| routes incoming internet traffic to the correct internal microservices | |
| use dynamic configurations. | |
| # Storage | |
| + Rook | |
| self-managing services for distributed storage systems (like Ceph) within k8s. | |
| + Longhorn | |
| highly available, lightweight persistent block storage for containers | |
| # Security | |
| + Falco | |
| detect unexpected, anomalous, or malicious behavior in real-time. | |
| + Cert-Manager | |
| manages TLS certificates for applications | |
| + Trivy | |
| scans container images and k8s conf for hidden vulnerabilities/misconfigurations. | |
| # Local Development & Provisioning | |
| + Minikube | |
| lightweight, single-node k8s cluster | |
| + Kind (Kubernetes IN Docker) | |
| local k8s clusters using Docker containers | |
| + Kops | |
| manage highly available/production-grade clusters in cloud | |
| # Developer Tooling | |
| + Lens | |
| IDE to manage, monitor clusters | |
| + Skaffold | |
| automates the workflow for deploying code to a cluster continuously | |
| # High Availability | |
| + Kube-vip | |
| --- HIGH AVAILABILITY | |
| # KUBE-VIP | |
| # ETCD topology - Replicated data storage (etcd) | |
| + stacked etcd | |
| etcd running on control plane nodes | |
| Control Plane Node 1: [API Server] [Controller Manager] [Scheduler] [etcd] | |
| Control Plane Node 2: [API Server] [Controller Manager] [Scheduler] [etcd] | |
| Control Plane Node 3: [API Server] [Controller Manager] [Scheduler] [etcd] | |
| - external etcd | |
| etcd on dedicated nodes | |
| Control Plane Node 1: [API Server] [Controller Manager] [Scheduler] | |
| Control Plane Node 2: [API Server] [Controller Manager] [Scheduler] | |
| Control Plane Node 3: [API Server] [Controller Manager] [Scheduler] | |
| etcd Node 1: [etcd] | |
| etcd Node 2: [etcd] | |
| etcd Node 3: [etcd] | |
| --- SECURITY | |
| --- DOC --- | |
| + node: physical/virtual server where run k8s (master, workers) | |
| + pod: instance, set of containers | |
| + service: make pod communication | |
| clusterIP | |
| nodePort | |
| + volume: stockage for pods, shared by containers | |
| persistent | |
| nonPersistent | |
| + deployment: manage deployment | |
| + namespace: subset of cluster, isolating set of service | |
| + operator: application-specific controller to manage complex workload (stateless) | |
| # k8s instanciation level | |
| + direct apply manifest with kubectl | |
| + helm chart | |
| - single chart for multiple apps | |
| - umbrella chart for templating | |
| + ArgoCD | |
| + CRD (Custom Resource Definition) | |
| --- KUSTOMIZE --- | |
| ## kustomize directory structure | |
| |k8s | |
| |-- base | |
| |-- overlays | |
| |-- ENV1 | |
| |-- ENV2 | |
| ## basic instructions | |
| apiVersion: kustomize.config.k8s.io/v1beta1 | |
| kind: Kustomization | |
| resources: | |
| - namespace.yaml | |
| - serviceaccount.yaml | |
| - pvc.yaml | |
| - rbac.yaml | |
| - service.yaml | |
| - deployment.yaml | |
| ## build | |
| $ kustomize build DIR/ | |
| ## build & deploy | |
| $ kustomize build DIR/ | kubectl apply -f - | |
| --- MISC --- | |
| ## run shell on container | |
| $ kubectl exec --stdin --tty CONTAINER -- /bin/bash | |
| ## restart an workload/deployment | |
| $ kubectl rollout restart deployment DEPLOYMENT | |
| --- MINIKUBE --- | |
| ## install | |
| https://odsc.medium.com/how-to-create-a-kubernetes-cluster-using-minikube-ed46e3453c64 | |
| ## profile/cluster | |
| + Add/create cluster | |
| $ minikube start -p CLUSTER | |
| ~$ minikube start -p CLUSTER --memory=10240 --cpus=4 --disk-size=30g --driver=docker --ports=8080:80 --ports=8443:443 --ports=30080:30080 --ports=30443:30443 | |
| + List cluster | |
| $ minikube config list profile | |
| + Activate cluster | |
| $ minikube profile CLUSTER | |
| ~$ kubectl config use-context CLUSTER | |
| + Get clusters | |
| $ minikube config get profile | |
| ~$ kubectl config get-contexts | |
| + Delete cluster | |
| $ minikube delete --profile CLUSTER | |
| ## activate addons | |
| $ minikube addons enable ingress | |
| $ minikube addons enable metrics-server | |
| --- LINT --- | |
| # manifest check/validation/syntax-check | |
| - kubeconform | |
| $ kubeconform -strict -summary -debug -verbose | |
| - kubescore | |
| $ kube-score score | |
| # dry-run | |
| $ kubectl apply -f manifest.yml --dry-run=client --validate=true | |
| --- SECRET ---- | |
| # declaration | |
| + secret | |
| apiVersion: v1 | |
| kind: Secret | |
| metadata: | |
| name: SECRET-NAME | |
| type: Opaque | |
| Data: | |
| KEY1: VALUE | |
| KEY2: VALUE | |
| + external secret | |
| (fetch from external vault (Vault, GCP/AWS Secret Manager)) | |
| - secret store connection | |
| apiVersion: external-secrets.io/v1beta1 | |
| kind: SecretStore | |
| metadata: | |
| name: secretStore-name | |
| spec: | |
| provider: | |
| vault: | |
| - get secret | |
| apiVersion: external-secrets.io/v1beta1 | |
| kind: ExternalSecret | |
| metadata: | |
| name: secret-name | |
| spec: | |
| refreshInterval: 1h | |
| secretStoreRef: | |
| name: secretStore-name | |
| kind: SecretStore | |
| target: | |
| name: native-secret-name | |
| data: | |
| - secretKey: KEY | |
| remoteRef: | |
| key: secret-path/name | |
| version: latest | |
| # usage | |
| spec: | |
| containers: | |
| - name: NAME | |
| image: IMAGE | |
| env: | |
| - name: VAR_NAME | |
| valueFrom: | |
| secretKeyRef: | |
| name: SECRET-NAME | |
| key: KEY_NAME | |
| # type | |
| - Data: base64 | |
| - stringData: plain text | |
| --- STORAGE/VOLUME | |
| # volume (PV/PVC) access modes | |
| - ReadWriteOnce (RWO) | |
| mounted as read-write by a single node at a time | |
| FOR standard databases (PostgreSQL, MySQL) where only one instance on one node should modify the data to prevent corruption | |
| - ReadOnlyMany (ROM) | |
| mounted as read-only by many nodes simultaneously | |
| FOR serve static web assets, shared dir/datasets | |
| - ReadWriteMany (RWX) | |
| mounted as read-write by many nodes at the same time | |
| FOR shared file systems (requires network-attached storage backends like NFS) | |
| - ReadWriteOncePod (RWOP) | |
| mounted as read-write by a single Pod in the entire cluster | |
| FOR strict concurrency applications where you want to ensure no other pod | |
| --- ISSUE --- | |
| ## pv stuck terminating | |
| $ kubectl patch pv PV_NAME -p '{"metadata":{"finalizers":null}}' | |
| or | |
| $ kubectl edit pv PV_NAME | |
| DELETE -> finalizers: - kubernetes.io/pv-protection | |
| $ kubectl delete pv PV_NAME --grace-period=0 --force | |
| --- OPERATOR | |
| # CRD (Custom Resource Definition) | |
| extend k8s api to use new object | |
| # operator | |
| specific controller, manage a CRD (Custom Ressouces Definition) | |
| # controller (control loop) | |
| - deployment controller (I want X pods) | |
| - job controller | |
| # example: | |
| + cert-manager | |
| Certificate, ClusterIssuer --> Let's Encrypt | |
| + External Secrets Operator (ESO) | |
| ExternalSecret, SecretStore --> Vault, SecretManager | |
| + Prometheus Operator | |
| ServiceMonitor, PodMonitor, PrometheusRule | |
| + Argo CD | |
| Application, AppProject | |
| + CloudNativePG | |
| Cluster, Pooler, Backup, ScheduledBackup --> PostgreSQL | |
| + KEDA (Kubernetes Event-driven Autoscaling) | |
| ScaledObject, ScaledJob | |
| + Flux CD | |
| GitRepository, HelmRelease, Kustomization | |
| + Traefik | |
| IngressRoute, Middleware | |
| + Velero | |
| Backup, Restore, Schedule | |
| + Cluster API (CAPI) | |
| Cluster, Machine, MachineDeployment | |
| + Istio | |
| VirtualService, DestinationRule, Gateway | |
| --- CRONJOB | |
| # create cronjob | |
| apiVersion: batch/v1 | |
| kind: CronJob | |
| metadata: | |
| name: MY-CRONJOB | |
| spec: | |
| schedule: "*/4 * * * *" | |
| jobTemplate: | |
| spec: | |
| template: | |
| spec: | |
| containers: | |
| - name: MY-CONTAINER | |
| image: httpd:latest | |
| command: | |
| - /bin/sh | |
| - -c | |
| - MY-COMMAND | |
| restartPolicy: OnFailure | |
| --- PODS | |
| # sidecar container | |
| (secondary container in a pod) | |
| - log collector | |
| - database auth proxy | |
| - service mesh data plane (Envoy/ Istio) | |
| # resource limits | |
| apiVersion: v1 | |
| kind: Pod | |
| metadata: | |
| name: httpd-pod | |
| spec: | |
| containers: | |
| - name: httpd-container | |
| image: httpd:latest | |
| resources: | |
| requests: | |
| memory: "15Mi" | |
| cpu: "100m" | |
| limits: | |
| memory: "20Mi" | |
| cpu: "100m" | |
| --- ROLE/PERMISSION/IAM | |
| # RBAC | |
| + Role | |
| (namespace) -> RoleBinding | |
| + ClusterRole | |
| (cluster) -> ClusterRoleBinding | |
| /!\ GCP: Workload Identity (GSA <-> KSA) + RBAC | |
| --- REPLICA SET | |
| # create replica set | |
| apiVersion: apps/v1 | |
| kind: ReplicaSet | |
| metadata: | |
| name: MY-REPLICASET | |
| labels: | |
| app: MY-APP | |
| type: FRONTEND | |
| spec: | |
| replicas: 4 | |
| selector: | |
| matchLabels: | |
| app: MY-APP | |
| type: FRONTEND | |
| template: | |
| metadata: | |
| labels: | |
| app: MY-APP | |
| type: FRONTEND | |
| spec: | |
| containers: | |
| - name: MY-CONTAINER | |
| image: httpd:latest | |
| --- ROLLOUT | |
| $ kubectl rollout history deployment/nginx-deployment | |
| $ kubectl rollout status deployment/nginx-deployment | |
| $ kubectl rollout undo deployment/nginx-deployment | |
| $ kubectl rollout undo deployment/nginx-deployment --to-revision=2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## dir structure | |
| |-- base | |
| |-- overlays | |
| |-- prd | |
| |-- ppr | |
| |-- int |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- OPEN SOURCE --- | |
| ## Apache | |
| ## Artistic License | |
| ## BSD | |
| ## Common Development and Distribution License (CDDL) | |
| ## CREATIVE COMMONS (CC) | |
| Licenses offering creators flexibility in sharing their work with the world while allowing them to choose the level of control they want over how their work is used, attributed, and shared | |
| https://creativecommons.org/share-your-work/cclicenses/ | |
| + CC BY (Attribution): | |
| allows others to distribute, remix, adapt, and build upon your work, even for commercial purposes, as long as they give you appropriate credit. | |
| This is the most permissive CC license. | |
| + CC BY-SA (Attribution-ShareAlike): | |
| Similar to CC BY but requires any derivative works to be licensed under the same terms, | |
| ensuring that subsequent creations remain open and shareable. | |
| + CC BY-NC (Attribution-NonCommercial): | |
| permits others to use and modify your work for non-commercial purposes, as long as they provide proper attribution. | |
| Commercial use is not allowed. | |
| + CC BY-ND (Attribution-NoDerivatives): | |
| others can reuse your work for any purpose, even commercially, but they can't make any modifications or derivative works. | |
| They must credit you as the creator. | |
| + CC BY-NC-SA (Attribution-NonCommercial-ShareAlike): | |
| combines the requirements of CC BY-NC and CC BY-SA, | |
| allowing for non-commercial use and requiring derivative works to be licensed under the same terms. | |
| + CC BY-NC-ND (Attribution-NonCommercial-NoDerivatives): | |
| the most restrictive CC license. | |
| allows others to download your work and share it with proper attribution, but they can't change it or use it for commercial purposes. | |
| + CC 0 (No Rights Reserved): | |
| not technically a license but a dedication to the public domain. | |
| waive all your copyright and related rights, effectively placing the work in the public domain | |
| ## Eclipse Public License (EPL) | |
| ## GNU General Public License (GPL): | |
| ## GNU Lesser General Public License (LGPL) | |
| ## MIT | |
| ## Mozilla Public License (MPL) | |
| --- CLOSED SOURCE --- | |
| --- COMMERCIAL --- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # manage arbitrary decision | |
| - what you undergo is not your fault, what you do with is your responsability |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- DOMAINS | |
| --- USERS | |
| ## allow users self-reset password | |
| -> Activate on Azure AD | |
| https://entra.microsoft.com/#view/Microsoft_AAD_IAM/PasswordResetMenuBlade/~/Properties |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC --- | |
| ## error! | |
| >> 1418 (HY000) at line 10185: This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary logging is enabled (you *might* want to use the less safe log_bin_trust_function_creators variable) | |
| 1- Execute: | |
| SET GLOBAL log_bin_trust_function_creators = 1; | |
| or | |
| 2- Add to mysql.ini config file: | |
| log_bin_trust_function_creators = 1; | |
| ## show mysqld startup config | |
| > SHOW VARIABLES; | |
| > SHOW VARIABLES LIKE '%innodb_log_buffer_size%'; | |
| --- MYSQL DUMP --- | |
| ## error mysqldump! | |
| >> Unknown table 'COLUMN_STATISTICS' in information_schema | |
| Flag "COLUMN_STATISTICS" is enabled by default in mysqldump 8 | |
| Run mysqldump with additional parameter: | |
| $ mysqldump --column-statistics=0 | |
| Make change permanent: | |
| --> ~/.my.cnf | |
| [mysqldump] | |
| column-statistics=0 | |
| --- TRANSACTION MODE --- | |
| ## start | |
| START TRANSACTION; | |
| CMD_1; | |
| CMD_N; | |
| ... | |
| ## commit | |
| COMMIT; | |
| ## rollback | |
| ROLLBACK; | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- DOC --- | |
| # OSI model | |
| PDNTSPA: Please Do Not Throw Sausage Pizza Away (Physical, Data Link, Network, Transport, Session, Presentation, Application)) | |
| + L7: Application | |
| Provides network services to end-user applications (web browsers, email clients) | |
| Protocols: HTTP, HTTPS, FTP, SMTP, DNS. | |
| Data Unit: Data | |
| + L6: Presentation | |
| Translates, formats, encrypts, and compresses data between Application layer apps | |
| Protocols: SSL/TLS (encryption), JPEG, ASCII, MIDI. | |
| Data Unit: Data | |
| + L5: Session | |
| Manages connections/authent between local and remote applications | |
| Protocols: NetBIOS, RPC, PPTP. | |
| Data Unit: Data | |
| + L4: Transport | |
| end-to-end communication between devices. | |
| Protocols: TCP/UDP | |
| Data Unit: Segments (TCP), Datagrams (UDP) | |
| + L3: Network | |
| routing data between different networks | |
| Protocols: IPv4, IPv6, ICMP, IPSec, Routers. | |
| Data Unit: Packets | |
| + L2: Data Link | |
| data transfer between two devices on the same network. | |
| Protocols: Ethernet, Wi-Fi, MAC, Network Switches. | |
| Data Unit: Frames | |
| + L1: Physical | |
| physical connection between the devices, transmits raw, unstructured data bits (1, 0) | |
| Protocols: Ethernet, fiber optics, radio waves, network hubs, repeaters. | |
| Data Unit: Bits | |
| # TCP/IP model | |
| + L4: Application Layer | |
| protocols communication between software applicationss over the network | |
| Protocols: HTTP/HTTPS, SMTP/IMAP, DNS, FTP | |
| Data Unit: Data (or Message) | |
| + L3: Transport Layer | |
| manage end-to-end communication between networks | |
| Protocols: TCP,UDP | |
| Data Unit: Segments (TCP), Datagrams (UDP) | |
| + L2: Internet Layer | |
| addressing and routing | |
| Protocols: IPv4, IPv6, ICMP | |
| Data Unit: Packets | |
| + L1: Network Access Layer | |
| Link Layer, Data Link and physical | |
| Protocols: Ethernet, Wi-Fi, MAC, fiber optic, NICs. | |
| Data Unit: Frames (transmitted as raw Bits) | |
| # Dynamic routing protocols | |
| + RIP (Routing Information Protocol) | |
| (obsolete) routes traffic based strictly on the lowest "hop count" (number of routers) | |
| + OSPF (Open Shortest Path First) | |
| maps the entire network and calculates the fastest route based on link bandwidth. | |
| + EIGRP (Enhanced Interior Gateway Routing Protocol) | |
| Cisco-designed protocol | |
| calculates paths using a mix of bandwidth and delay, keeping backup routes ready for near-instant failover | |
| + IS-IS (Intermediate System to Intermediate System) | |
| as OSPF but operating at Layer 2 | |
| for the internal backbones of massive ISPs and telecom carriers | |
| + BGP (Border Gateway Protocol) | |
| core protocol of the global Internet | |
| route traffic between organizations and ISPs based on administrative rules, policies, and business agreements | |
| --- IP ADDRESS --- | |
| ## address classes | |
| A: 0.0.0.0 - 126.255.255.255 (private: 10.0.0.0 - 10.255.255.255) | |
| B: 128.0.0.0 - 191.255.255.255 (private: 172.16.0.0 - 172.31.255.255) | |
| C: 192.0.0.0 - 223.255.255.255 (private: 192.168.1.0 - 192.168.255.255) | |
| D: 224.0.0.0 - 239.255.255.255 (for multicast) | |
| E: 240.0.0.0 - 255.255.255.255 (reserved by IETF) | |
| 127.0.0.0: for local loop | |
| 0.0.0.0: default route | |
| -- IPV6 --- | |
| ## disable IPv6 | |
| $ sudo cat > /etc/sysctl.d/70-disable-ipv6.conf | |
| net.ipv6.conf.all.disable_ipv6 = 1 | |
| $ sudo sysctl -p -f /etc/sysctl.d/70-disable-ipv6.conf |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC --- | |
| ## misc | |
| Check syntax | |
| $ nginx -t | |
| Dump all config | |
| $ nginx -T | |
| Reload conf | |
| $ nginx -s reload | |
| --- BUFFER --- | |
| ## tuning buffer | |
| ERROR: upstream sent too big header while reading response header from upstream | |
| --> https://www.getpagespeed.com/server-setup/nginx/tuning-proxy_buffer_size-in-nginx | |
| proxy_busy_buffers_size 24k; | |
| proxy_buffers 64 4k; | |
| proxy_buffer_size 16k; | |
| --- PARSE LOG --- | |
| ## log format (access.log / error.log) | |
| --> $remote_addr $RFC_1413_identity $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" | |
| 172.17.0.1 - user [06/Aug/2024:14:55:37 +0000] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (X11; Linux x86_64)" "-" | |
| ## parsing | |
| + Sort access by response code | |
| $ cat access.log | cut -d '"' -f3 | cut -d ' ' -f2 | sort | uniq -c | sort -rn | |
| $ awk '{print $9}' access.log | sort | uniq -c | sort -rn | |
| 210433 200 | |
| 1144 502 | |
| 706 404 | |
| + Find specific code (404/502) | |
| $ awk '($9 ~ /404/)' access.log | awk '{print $7}' | sort | uniq -c | sort -rn | |
| 21 /SUBLINKSLINKS/SUBLINKS/ENDPOINT | |
| 19 /SUBLINKS/robots.txt | |
| + Address who return a specific code | |
| $ awk -F\" '($2 ~ "/SUBLINKS/ENDPOINT"){print $1}' access.log | awk '{print $1}' | sort | uniq -c | sort -r |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## error: ngrok - install failed [Error: EACCES: permission denied | |
| --> https://github.com/bubenshchykov/ngrok#usage | |
| $ npm install --unsafe-perm -g ngrok |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC --- | |
| build base image | |
| ## Do not remove instance when packer build fails | |
| $ packer build -on-error=abort APP.json |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- JQ --- | |
| ## get all paths | |
| $ jq -r '[paths | join(".")]' FILE.json | |
| ## get all paths and its value | |
| $ jq --stream -r 'select(.[1]|scalars!=null) | "\(.[0]|join(".")): \(.[1]|tojson)"' FILE.json | |
| $ jq -r 'paths(scalars) as $p | [ ( [ $p[] | tostring ] | join(".") ), ( getpath($p) | tojson )] | join(": ")' FILE.json |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- BASH | |
| abcd contains | |
| 'abcd' exact match | |
| ^abcd start with | |
| .abcd$ end by | |
| !abcd not include | |
| !^abcd not start with | |
| !.abcd$ not end with |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| FORMATION | |
| RTE: | |
| Release Train Engineer | |
| STE: | |
| Solution Train Engineer |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC --- | |
| ## generate random password | |
| Best Practice -> N=16 | |
| $ openssl rand -base64 N | |
| $ head -c N /dev/urandom | base64 | |
| $ date +%s | sha256sum | base64 | head -c N | |
| $ uuidgen | tr -d '-' | head -cN | |
| $ gpg --gen-random --armor 1 N | |
| $ dd if=/dev/urandom bs=1 count=N 2>/dev/null | base64 | |
| $ python3 -c "import secrets; print(secrets.token_urlsafe(N))" | |
| ## cat file witout comments lines | |
| $ cat FILE | grep -v '^#' | |
| $ grep -v '^#' FILE | |
| Match also blank lines | |
| $ grep -Ev '(^#|^$)' FILE | |
| ## copy/move all files except some | |
| $ cp|mv !(FILE_1|FILE_N) DEST/ | |
| ## list all functions defined in shell | |
| See list | |
| $ declare -F | |
| $ compgen -A function | |
| See list of function + aliases | |
| $ compgen -a -A function | |
| See list with content | |
| $ declare -f | |
| See content of a fuction | |
| $ type FUNCTION | |
| ## redirect cmd output/errors to file | |
| $ CMD --params 2>&1 | tee /path/to/file.log | |
| ## chaining command with list output | |
| $ CMD_1_OUTPUT_LIST | xargs -I {} CMD_2_PROCESS_EVERY_ITEM_OF_THE_LIST | |
| ## update-alternatives | |
| Install | |
| $ update-alternatives --install /path/to/link name /path/to/binary [priority] | |
| $ update-alternatives --install /usr/bin/python python /usr/bin/python2.7 2 | |
| $ update-alternatives --install /usr/bin/python python /usr/bin/python3.7 3 | |
| Select | |
| $ update-alternatives --config python | |
| ## XML lint, formatting | |
| in package: libxml2-utils | |
| $ xmllint --format UGLY.xml --output FORMATTED.xml | |
| ## sudo without password | |
| $ visudo | |
| %sudo ALL=(ALL) NOPASSWD:ALL | |
| ## autocompletion | |
| --> in bashrc | |
| if [ -f /etc/bash_completion ]; then | |
| . /etc/bash_completion | |
| fi | |
| ## add timestamp to bash_history | |
| $ export HISTTIMEFORMAT="%F %T " | |
| # cat > /etc/profile.d/timestamp-history.sh | |
| HISTTIMEFORMAT="%F %T " | |
| --- ARCHIVE --- | |
| ## zip multiple files in one archive | |
| $ zip ARCHIVE.zip FILE1 FILE2 FILEN $FILE* | |
| $ zip -r ARCHIVE.zip DIR/ | |
| ## zip/unzip with password | |
| $ zip -e -P "PASSWORD" FILE FILE.zip | |
| $ zip -e -P "PASSWORD" FILE FILE.zip | |
| ## list file in zip | |
| $ unzip -l oc-4.13.0.zip "*/" | |
| $ zipinfo -1 oc-4.13.0.zip "*/" | |
| --- GRUB --- | |
| ## change default grub selection | |
| -> /etc/default/grub | |
| GRUB_DEFAULT=0 | |
| --- SERVICE --- | |
| ## /etc/init.d | |
| $ sudo vim /etc/init.d/SERVICE | |
| --> https://gist.githubusercontent.com/miromannino/a17f3e6f3fdcb4d94a1f/raw/e9bc2a8179860701224be896b420a635454dd3bd/service.sh | |
| $ sudo chmod a+x /etc/init.d/SERVICE | |
| $ sudo update-rc.d SERVICE defaults | |
| --- LOGIN MESSAGE | |
| ## MOTD: login banner message for system-wide announcements | |
| /etc/motd | |
| ## HUSHLOGIN: login banner message for individual user (suppress motd) | |
| ~/.hushlogin | |
| --- HARDWARE --- | |
| ## check hardware device informations | |
| List all | |
| $ lspci -s | |
| ID Type: Fabricant, Version | |
| 0000:2a:00.0 Network controller: Realtek Semiconductor Co., Ltd. Device b852 | |
| Show details info about a device | |
| $ lspci -v -s ID | |
| --- FILE --- | |
| ## delete line from N file to the end | |
| $ sed -i 'N,$d' FILE | |
| --- GITHUB --- | |
| ## ssh access via https | |
| https://docs.github.com/en/authentication/troubleshooting-ssh/using-ssh-over-the-https-port | |
| Test connection | |
| $ ssh -T -p 443 git@ssh.github.com | |
| Clone | |
| $ git clone ssh://git@ssh.github.com:443/USERNAME/REPOSITORY.git | |
| --- DESKTOP MANAGER --- | |
| ## change wallpaper | |
| KDE | |
| $ qdbus org.kde.plasmashell /PlasmaShell org.kde.PlasmaShell.evaluateScript 'var allDesktops = desktops();print (allDesktops);for (i=0;i<allDesktops.length;i++) {d = allDesktops[i];d.wallpaperPlugin = "org.kde.image";d.currentConfigGroup = Array("Wallpaper", "org.kde.image", "General");d.writeConfig("Image", "file:///PATH/TO/FILE.png")}' | |
| --- SERVICES --- | |
| ## check service journalctl | |
| $ journalctl -u SERVICE.service | |
| --- PACKAGE MANAGER --- | |
| ## check package who provided a CMD/file | |
| $ dpkg -S $(which CMD) | |
| $ yum whatprovides $(which CMD) | |
| $ dnf provides $(which CMD) | |
| $ rpm -qf $(which CMD | |
| ~~ SNAP | |
| ## snap | |
| $ snap install PACKAGE | |
| $ snap info PACKAGE | |
| $ snap refresh PACKAGE --channel=X.Y/stable --classic | |
| ~~ APT | |
| ## use apt behind proxy | |
| --> /etc/apt/apt.conf.d/0-proxy | |
| Acquire::http::Proxy "http://PROXY_URL"; | |
| Acquire::https::Proxy "https://PROXY_URL"; | |
| Acquire::ftp::Proxy "ftp://PROXY_URL"; | |
| ## bypass source.list certificate verification behind proxy (disables apt's OCSP verification) | |
| https://askubuntu.com/questions/1095266/apt-get-update-failed-because-certificate-verification-failed-because-handshake | |
| --> /etc/apt/apt.conf.d/99verify-peer.conf | |
| Acquire { https::Verify-Peer false } | |
| ## hold a specific version of package | |
| $ apt-mark hold PACKAGE-x.y.z | |
| or | |
| $ vim /etc/apt/preferences.d/PACKAGE | |
| Package: linux-image-4.19.0-6-amd64 | |
| Pin: version 4.19.81-2+deb10u1 | |
| Pin-Priority: -1 | |
| --- SSH --- | |
| ## ssh config (.ssh/config) | |
| + keep connection alive | |
| TCPKeepAlive yes | |
| ServerAliveInterval 30 | |
| + use private key | |
| IdentityFile ~/.ssh/key.openssh | |
| + log level | |
| Loglevel QUIET|FATAL|ERROR|VERBOSE|DEBUG|DEBUG1|DEBUG2|DEBUG3:-INFO | |
| ## verify ssh keypair match | |
| $ ssh-keygen -y -e -f <private_key> | |
| compare with <public_key> | |
| ## proxy tcp tunnel over ssh | |
| SSH only | |
| $ /usr/bin/ssh -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -NL [LOCAL_PORT]:127.0.0.1:[REMOTE_PORT] [REMOTE_USER]@[REMOTE_ADRESS] -p [REMOTE_SSH_PORT] | |
| AUTOSSH | |
| $ /usr/bin/autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -NL [LOCAL_PORT]:127.0.0.1:[REMOTE_PORT] [REMOTE_USER]@[REMOTE_ADRESS] -p [REMOTE_SSH_PORT] | |
| ssh tunnel for mysql | |
| $ ssh -N -L LOCAL_PORT:REMOTE_INTERFACE_IP:REMOTE_PORT USER@REMOTE_IP | |
| ## disable password auth for specific users/group | |
| --> /etc/ssh/sshd_config | |
| " | |
| #PasswordAuthentication yes | |
| Match [User|Group|Adress] [USER] | |
| PasswordAuthentication no | |
| " | |
| [USER]: - user1,user2 | |
| - group1,group2 | |
| - !root | |
| - 192.168.1.2,192.168.12.* | |
| - hostname | |
| - | |
| ## Without host key verification | |
| $ ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" user@host | |
| ## save key passphrase (avoid prompt) | |
| $ eval `ssh-agent -s` | |
| $ ssh-add ~/.ssh/PRIVATE_KEY | |
| ## remove (left blank) / change key passphrase | |
| $ ssh-keygen -p -P CURRENT_PASSPHRASE -N "NEW_PASSPHRASE|BLANK" PATH/TO/KEY | |
| --- PDF --- | |
| ## combine pdf | |
| $ pdfunite file1.pdf file2.pdf fileN.pdf output.pdf | |
| --- USER/GROUP --- | |
| ## add system user | |
| $ sudo adduser --group GROUP --system USER | |
| ~$ sudo useradd -g GROUP -d /dev/null -s /usr/sbin/nologin USER | |
| ## delete user from group | |
| $ gpasswd -d USER GROUP | |
| ## run as other user | |
| $ runuser -l USER -c 'cmd args' | |
| ~$ runuser -u USER -- cmd args | |
| ~$ su - USER -c "cmd args" | |
| ~$ sudo -u USER cmd args | |
| --- DISK | |
| # disk space | |
| space left | |
| $ df -Th | |
| disk usage | |
| $ du -sh --max-depth=1 | |
| # inspect disk | |
| disk layout | |
| $ lsblk -f | |
| partition table | |
| $ fdisk -l | |
| # disk perf | |
| $ iostat -xz -1 | |
| $ iotop | |
| # health check | |
| $ smartctl | |
| $ dmesg | grep -i error | |
| --- LINK --- | |
| ## change symbok link target | |
| $ readlink -v LINK | |
| $ ln -sfn /PATH/TO/NEW/FILE LINK_NAME | |
| --- XARGS --- | |
| ## loop with xargs | |
| $ $(CMD TO GET A LIST) |xargs -I {} $(CMD TO LOOP ON THE LIST) {} | |
| --- OPENSSL --- | |
| [GET INFO] | |
| ## check CSR (Certificate Signing Request) | |
| $ openssl req -text -noout -verify -in CSR.csr | |
| /!\ CSR FIELDS ACCRONYM | |
| C: Country Name: | |
| 2-letter ISO 3166-1 alpha-2 code (e.g., US for United States) | |
| ST: State or Province Name: | |
| Full name of the state or province (e.g., California) | |
| L: Locality Name: | |
| City or town where the organization is located (e.g., San Francisco) | |
| O: Organization Name: | |
| Legal name of the company or organization (e.g., Example Inc.) | |
| OU: Organizational Unit Name: | |
| Department within the organization (e.g., IT Department) | |
| CN: Common Name: | |
| For websites: The fully qualified domain name (FQDN) (e.g., www.example.com) | |
| For other entities: A unique identifier (e.g., server name, email address) | |
| ## check Private Key | |
| $ openssl rsa -in privateKey.key -check | |
| ## check PEM formatted certificate | |
| $ openssl x509 -text -noout -in cert.{pem|crt} | |
| ## check DER formatted certificate | |
| $ openssl x509 -in MYCERT.der -inform der -text | |
| ## check PKCS#12 file (.pfx or .p12) | |
| $ openssl pkcs12 -info -in keyStore.p12 | |
| ## check certificate from URL | |
| $ openssl s_client -showcerts -connect [URL]:443 | |
| ## check expiration date | |
| $ echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates | |
| ## Check all SAN (Subject Alternative Name) in the certificate | |
| $ openssl s_client -connect website.com:443 | openssl x509 -noout -text | grep DNS: | |
| ## check if cert & private key match | |
| $ openssl x509 -modulus -noout -in CERT.crt |md5sum | |
| $ openssl rsa -modulus -noout -in PRIV.priv |md5sum | |
| [GENERATE CERTIFICATE] | |
| ## Generate PEM file from key | |
| $ openssl rsa -in FILE.key -outform PEM -out FILE.pem | |
| ## generate Private Key and Certificate Signing Request | |
| $ $ openssl req -nodes -newkey rsa:2048 -sha256 -keyout PRIVATE.key -out CSR.csr -utf8 | |
| ## generate self-signed certificate | |
| $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout privateKey.key -out certificate.crt | |
| ## generate CSR for existing private key | |
| $ openssl req -out CSR.csr -key privateKey.key -new | |
| ## generate CSR based on an existing certificate | |
| $ openssl x509 -x509toreq -in certificate.crt -out CSR.csr -signkey privateKey.key | |
| ## Remove passphrase from Private Key | |
| $ openssl rsa -in privateKey.pem -out newPrivateKey.pem | |
| ## Generate private key and self-signed certificate for localhost | |
| $ openssl req -x509 -out localhost.crt -keyout localhost.key \ | |
| -newkey rsa:2048 -nodes -sha256 \ | |
| -subj '/CN=localhost' -extensions EXT -config <( \ | |
| printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth") | |
| ## Generate certificate CRT from CSR and private key | |
| $ openssl x509 -req -in CSR.csr -signkey PRIVATE.key -out CERT.crt | |
| [CONVERTING] | |
| ## Convert DER file (.crt .cer .der) to PEM | |
| $ openssl x509 -inform der -in certificate.cer -out certificate.pem | |
| ## Convert PEM file to DER | |
| $ openssl x509 -outform der -in certificate.pem -out certificate.der | |
| ## Convert PKCS#12 file (.pfx .p12) containing a private key and certificates to PEM | |
| $ openssl pkcs12 -in keyStore.pfx -out keyStore.pem -nodes | |
| You can add -nocerts to only output the private key or add -nokeys to only output the certificates. | |
| ## Convert PEM certificate file and a private key to PKCS#12 (.pfx .p12) | |
| $ openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt -certfile CACert.crt | |
| --- MEDIA --- | |
| ## convert image with IMAGEMAGICK | |
| --> install imagemagick | |
| Change image format | |
| $ convert FILE_SRC.jpg FILE_DEST.png | |
| Reduce image quality | |
| $ convert FILE_SRC.jpg -quality X% FILE_DEST.jpg | |
| Resize image | |
| $ convert FILE_SRC.jpg -resize 200x300 FILE_DEST.jpg | |
| --- NETWORK --- | |
| # test port | |
| $ telnet | |
| $ nc | |
| $ nmap | |
| With pseudo-device files | |
| $ </dev/tcp/HOST/PORT" && echo "OK: Port open" || echo "KO: Port closed" | |
| or | |
| $ timeout 3 bash -c "</dev/tcp/HOST/PORT" && echo "OK: Port open" || echo "KO: Port closed" | |
| $ HOST=HOST; PORT=PORT; timeout 3 bash -c "</dev/tcp/$HOST/$PORT" && echo "$? - $HOSTNAME -> $HOST:$PORT - open" || echo "$? - $HOSTNAME -> $HOST:$PORT - closed" | |
| TRACK PACKET ON PORT | |
| + server (wait) | |
| $ tcpdump -i any port 1234 | |
| + client (send) | |
| $ nc -v -u 11.22.33.44 1234 | |
| # wget | |
| pass user/pwd | |
| $ wget --http-user USER --http-password PWD URL | |
| ## tcpdump | |
| show interface | |
| $ tcpdump -D | |
| on specific interface | |
| $ tcpdump -i INTERFACE | |
| on specific port | |
| $ tcpdump port PORT -i INTERFACE | |
| capture only N number of packets | |
| $ tcpdump -c N -i INTERFACE | |
| display captured packets in ASCII | |
| $ tcpdump -A -i INTERFACE | |
| display Captured Packets in HEX and ASCII | |
| $ tcpdump -XX -i INTERFACE | |
| capture and save packets in a file | |
| $ tcpdump -w FILE.pcap -i INTERFACE | |
| read captured packets file | |
| $ tcpdump -r FILE.pcap | |
| capture IP address packets | |
| $ tcpdump -n -i INTERFACE | |
| capture only TCP Packets | |
| $ tcpdump -i INTERFACE tcp | |
| ## netcat | |
| + Transfert file | |
| --> on receiver | |
| $ nc -l -p PORT > FILE | |
| --> on sender | |
| $ nc -w 3 DEST_IP PORT < FILE | |
| + Compress file & transfert | |
| --> on receiver | |
| $ nc -l -p PORT | uncompress -c | tar xvfp - | |
| --> on sender | |
| $ tar cfp - DIR | compress -c | nc -w 3 DEST_IP PORT | |
| + Reverse shell with root prilileges | |
| --> target | |
| $ ncat -l PORT -e /bin/bash -v -w 5000ms | |
| --> source | |
| $ ncat 127.0.0.1 PORT -v | |
| + Test open port | |
| $ nc -vn IP PORT | |
| $ nc -vnz -w IP RANGE-IP | |
| # scan wifi | |
| $ iwlist INTERFACE scan | |
| connect to wifi via cli | |
| $ nmcli dev wifi connect SSID password PASSWORD | |
| # cURL | |
| + API testing | |
| GET | |
| $ curl URL/api/ressource | |
| POST | |
| $ curl -X POST -H "Content-Type: application/json" -d '{"key":"value"}' URL/api | |
| Ignore SSL cert | |
| $ curl -k URL | |
| + connect test | |
| Follow redirect | |
| $ curl -I -L URL | |
| Test with proxy | |
| $ curl -x http://proxy-server:8080 URL | |
| # download | |
| $ curl -o OUTPUT_FILE URL/REMOTE_FILE | |
| # upload | |
| $ curl -T LOCAL_FILE URL/dir | |
| --- MONITORING | |
| # CPU | |
| $ top/htop | |
| $ mpstat -P ALL 1 | |
| $ vmstat 1 | |
| $ pidstat -u UID | |
| # memory | |
| $ free -mh | |
| $ smem -sort pss | |
| $ pmap -x PID | |
| $ swapon --show |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- MISC --- | |
| # shebang/ hashbang | |
| #!/bin/sh|bash | |
| #!/usr/bin/env bash|python | |
| # shell options | |
| set -o pipefail | |
| set -o errexit | |
| set -o nounset | |
| set -o xtrace | |
| # include debug option in run | |
| DEBUG=${DEBUG:=0} | |
| [[ $DEBUG -eq 1 ]] && set -o xtrace | |
| # redirect script output to stdout & in log file | |
| exec &> >(tee -a "/PATH/TO/FILE.LOG") | |
| ## prevent script to be executed (library case) | |
| if [ "${0}" == "${BASH_SOURCE[0]}" ]; then | |
| echo "This script is not meant to be executed directly!" | |
| echo "Use this script only by sourcing it." | |
| exit 1 | |
| fi | |
| ## trap event | |
| trap 'cmd' EVENT1 EVENT2 EVENT3 | |
| e.g: trap 'exit' INT TERM ERR | |
| --- VARIABLES --- | |
| ## get current directory | |
| DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | |
| ## array, list | |
| ARRAY=('VALUE_1' 'VALUE_2' 'VALUE_N') | |
| ARRAY=([NAME_1]='VALUE_1' [NAME_2]='VALUE_2' [NAME_N]='VALUE_N') | |
| declare -a ARRAY | |
| ARRAY[NAME_1]="VALUE_1" | |
| ARRAY[NAME_2]="VALUE_2" | |
| ARRAY[NAME_N]="VALUE_N" | |
| Get all values: | |
| ${ARRAY[@]} | |
| Get all names in index: | |
| ${!ARRAY[@]} | |
| ## parameter Substitution | |
| --> http://www.tldp.org/LDP/abs/html/parameter-substitution.html#PARAMSUBREF | |
| --> https://www.cyberciti.biz/tips/bash-shell-parameter-substitution-2.html | |
| - if not set, use default | |
| VAR=${VAR_EXPECTED:-default_value} | |
| VAR=${VAR_EXPECTED:-`cmd`} | |
| -> VAR=default_value | |
| Double assignment: | |
| VAR=${VAR_EXPECTED:=default_value} | |
| -> VAR=VAR_EXPECTED=default_value | |
| - if not set, print error message and exit 1 | |
| VAR=${VAR_EXPECTED:?ERROR VAR_EXPECTED is not defined} | |
| - replace content | |
| a="b.c" | |
| d="${a/./-}" | |
| echo ${d} -> "b-c" | |
| --> ${my_var//?/*} = replace any single character by * | |
| - to uppercase | |
| VAR=${name^^} | |
| - to lowercase | |
| var=${VAR,,} | |
| - get last string of an URL | |
| ${VAR##*/} | |
| --- ARRAY --- | |
| ## declare array var | |
| declare -A my_var | |
| my_var["content1"]="value1" | |
| my_var["content2"]="value2; | |
| ## append item to array | |
| declare -a ARRAY | |
| ARRAY=("a" "b") | |
| ARRAY+=("c" "d) | |
| ## array length | |
| ${#ARRAY_NAME[@]} | |
| --- COLOR OUTPUT --- | |
| --> https://dev.to/ifenna__/adding-colors-to-bash-scripts-48g4 | |
| Color | Foreground Code | Background Code | |
| Black 30 40 | |
| Red 31 41 | |
| Green 32 42 | |
| Yellow 33 43 | |
| Blue 34 44 | |
| Gray 90 100 | |
| White 97 107 | |
| COLOR="\e[33m" | |
| COLOR_END="\e[0m" | |
| echo -e "${COLOR} TEXT ${COLOR_END}" | |
| --- CONDITIONING --- | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## disable system beep | |
| $ rmmod pcspkr; echo "blacklist pcspkr" >>/etc/modprobe.d/blacklist.conf | |
| --- GRUB --- | |
| ## change default grub selection | |
| -> /etc/default/grub | |
| GRUB_DEFAULT=0 | |
| --- SELINUX | |
| # Check current mode | |
| getenforce | |
| # Set temporary mode (until reboot) | |
| sudo setenforce 0 # Set to Permissive | |
| sudo setenforce 1 # Set to Enforcing | |
| # View detailed status | |
| sestatus | |
| # Troubleshoot denials | |
| sudo sealert -a /var/log/audit/audit.log | |
| # Manage booleans | |
| getsebool -a # List all booleans | |
| sudo setsebool -P httpd_can_network_connect on | |
| # SELinux config file | |
| /etc/selinux/config | |
| /var/log/audit/audit.log | |
| /etc/selinux/targeted/ (modules de stratégie ciblés) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## add trusted CA certificates | |
| install package "ca-certificates" | |
| $ cp CERT.cert /usr/local/share/ca-certificates/CERT.crt | |
| $ sudo /usr/local/share/ca-certificates/ | |
| ## get certificate from URL | |
| $ openssl s_client -showcerts -connect URL:443 </dev/null 2>/dev/null|openssl x509 -outform PEM > CERT.pem | |
| --- GET INFO --- | |
| ## check CSR (Certificate Signing Request) | |
| $ openssl req -text -noout -verify -in CSR.csr | |
| ## check Private Key | |
| $ openssl rsa -in privateKey.key -check | |
| ## check PEM formatted certificate | |
| $ openssl x509 -text -noout -in cert.{pem|crt} | |
| ## check DER formatted certificate | |
| $ openssl x509 -in MYCERT.der -inform der -text | |
| ## check PKCS#12 file (.pfx or .p12) | |
| $ openssl pkcs12 -info -in keyStore.p12 | |
| ## check certificate from URL | |
| $ openssl s_client -showcerts -connect [URL]:443 | |
| ## check expiration date | |
| $ echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates | |
| ## Check all SAN (Subject Alternative Name) in the certificate | |
| $ openssl s_client -connect website.com:443 | openssl x509 -noout -text | grep DNS: | |
| --- GENERATE CERTIFICATE --- | |
| ## generate Private Key and Certificate Signing Request | |
| $ openssl req -out CSR.csr -new -newkey rsa:2048 -nodes -keyout privateKey.key | |
| ## generate self-signed certificate | |
| $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout privateKey.key -out certificate.crt | |
| ## generate CSR for existing private key | |
| $ openssl req -out CSR.csr -key privateKey.key -new | |
| ## generate CSR based on an existing certificate | |
| $ openssl x509 -x509toreq -in certificate.crt -out CSR.csr -signkey privateKey.key | |
| ## Remove passphrase from Private Key | |
| $ openssl rsa -in privateKey.pem -out newPrivateKey.pem | |
| ## Generate private key and self-signed certificate for localhost | |
| $ openssl req -x509 -out localhost.crt -keyout localhost.key \ | |
| -newkey rsa:2048 -nodes -sha256 \ | |
| -subj '/CN=localhost' -extensions EXT -config <( \ | |
| printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth") | |
| --- CONVERTING USING OpenSSL --- | |
| ## Convert DER file (.crt .cer .der) to PEM | |
| $ openssl x509 -inform der -in certificate.cer -out certificate.pem | |
| ## Convert PEM file to DER | |
| $ openssl x509 -outform der -in certificate.pem -out certificate.der | |
| ## Convert PKCS#12 file (.pfx .p12) containing a private key and certificates to PEM | |
| $ openssl pkcs12 -in keyStore.pfx -out keyStore.pem -nodes | |
| You can add -nocerts to only output the private key or add -nokeys to only output the certificates. | |
| ## Convert PEM certificate file and a private key to PKCS#12 (.pfx .p12) | |
| $ openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt -certfile CACert.crt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Lifecycle | |
| - terraform init | |
| Downloads providers and initializes the backend | |
| - terraform fmt | |
| Rewrites code to canonical format | |
| - terraform validate | |
| Checks syntax errors | |
| - terraform plan | |
| "Dry run." Compares code vs. state and calculates changes (+ create, - destroy, ~ update) | |
| - terraform apply | |
| Executes | |
| - terraform destroy | |
| Deletes all resources tracked in the state file. | |
| # Linter | |
| $ terraform fmt | |
| $ terraform validate | |
| $ tflint | |
| # Importing Infrastructure | |
| add ressource created manually into terraform | |
| $ terraform import google_storage_bucket.my_bucket bucket-id | |
| # backend | |
| state file storage | |
| - local | |
| - remote | |
| # State Management | |
| terraform.tfstate file as source of truth | |
| + Remote Backend | |
| S3 (AWS) or GCS (GCP). | |
| Encryption, shared access, and Locking. | |
| + Workspaces | |
| Allow multiple state files from one set of code (Env dev, prod) | |
| /!\ In production, use separate directories (env/dev, env/prod) over workspaces to isolate failure domains strictly. | |
| + Drift | |
| When real-world infra was changed manualy and differs from the state file. | |
| + Fix | |
| terraform refresh/ fix drift (update the state to match reality) or terraform apply | |
| # backend best-practice | |
| - State Locking | |
| prevent two pipelines from running simultaneously on remote backend | |
| - State Encryption | |
| encrypt sensitive data | |
| - Version Control | |
| can rollback to a previous state file | |
| # Modules | |
| Reusable packages of Terraform code. | |
| + Root Module: working directory. | |
| + Child Module: module called by the root | |
| # workspace | |
| manage multiple distinct state files | |
| (for multiple env: terraform.tfstate.dev, terraform.tfstate.prod) | |
| $ terraform workspace new dev | |
| $ terraform workspace select dev | |
| Gotcha: | |
| + Workspaces | |
| Short-lived feature branches, temporary environment | |
| Exact replicas for env Dev, Stage, and Prod | |
| + Directory Separation (The Enterprise Standard): | |
| separate folders, code isolation, clarity, RBAC | |
| envs/ | |
| dev/ (main.tf, variables, backend config) | |
| prod/ (main.tf, variables, backend config) | |
| modules/ (Shared code) | |
| --- MISC | |
| # check pull status | |
| $ terraform state pull | |
| --- VARIABLE | |
| # output message | |
| output "MY_MESSAGE" { | |
| value = module.parameter.argument | |
| value = "value string" | |
| description = "Description of output" | |
| } | |
| # custom variable validation | |
| variable "environment" { | |
| type = string | |
| description = "The target deployment environment." | |
| validation { | |
| condition = contains(["dev", "staging", "prod"], var.environment) | |
| error_message = "The environment variable must be one of: dev, staging, or prod." | |
| } | |
| } | |
| --- PROVISIONERS | |
| # built-in provisioners | |
| + local-exec | |
| provisioner "local-exec" { | |
| command = "echo ${self.network_interface[0].access_config[0].nat_ip} >> line_ips.txt" | |
| } | |
| + remote-exec | |
| connection { | |
| type = "ssh" | |
| user = "ubuntu" | |
| private_key = file("~/.ssh/id_rsa") | |
| host = self.public_ip | |
| } | |
| provisioner "remote-exec" { | |
| inline = [ | |
| "sudo apt-get update" | |
| ] | |
| } | |
| + file | |
| connection { | |
| type = "ssh" | |
| user = "ubuntu" | |
| host = self.public_ip | |
| } | |
| provisioner "file" { | |
| source = "conf/nginx.conf" | |
| destination = "/etc/nginx/nginx.conf" | |
| } | |
| --- IP/ADRESSES | |
| # AWS | |
| + Create/allocate Elastic IP (EIP) | |
| resource "aws_eip" "MY-EIP" { | |
| domain = "vpc" | |
| tags = { | |
| Name = "MY-EIP" | |
| } | |
| } | |
| --- VPC | |
| # AWS | |
| resource "aws_vpc" "MY_VPC" { | |
| cidr_block = "192.168.0.0/24" | |
| enable_dns_support = true | |
| enable_dns_hostnames = true | |
| tags = { | |
| Name = "MY_VPC" | |
| } | |
| } | |
| --- KEY PAIR | |
| # AWS | |
| + create key pair | |
| resource "tls_private_key" "RSA-KEYL" { | |
| algorithm = "RSA" | |
| rsa_bits = 4096 | |
| } | |
| resource "aws_key_pair" "MY-KEY-PAIR" { | |
| key_name = "MY-KEY-PAIR" | |
| public_key = tls_private_key.RSA-KEY.public_key_openssh | |
| } | |
| --- INSTANCE | |
| # AWS | |
| + Get default security group | |
| data "aws_security_group" "DEFAULT-SG" { | |
| name = "default" | |
| } | |
| + create instance EC2 | |
| resource "aws_instance" "MY-INSTANCE-EC2" { | |
| ami = "ID_IMAGE" | |
| instance_type = "t2.micro" | |
| key_name = aws_key_pair.KEY-PAIR.key_name | |
| vpc_security_group_ids = [data.aws_security_group.DEFAULT-SG.id] | |
| tags = { | |
| Name = "MY-INSTANCE-EC2" | |
| } | |
| } | |
| + create AMI from instance | |
| resource "aws_ami_from_instance" "MY-INSTNACE-AMI" { | |
| name = "MY-INSTANCE-AMI" | |
| source_instance_id = aws_instance.MY-INSTANCE-EC2.id | |
| timeouts { | |
| create = "30m" | |
| } | |
| } | |
| --- SECRET | |
| # use external vault provider | |
| terraform { | |
| required_providers { | |
| vault = { | |
| source = "hashicorp/vault" | |
| version = "~> 4.0" | |
| } | |
| postgresql = { | |
| source = "cyrilgdn/postgresql" | |
| version = "~> 1.25" | |
| } | |
| } | |
| } | |
| provider "vault" { | |
| address = var.vault_address | |
| } | |
| data "vault_kv_secret_v2" "db_credentials" { | |
| mount = "secret" | |
| name = "production/database" | |
| } | |
| provider "postgresql" { | |
| host = "10.0.0.5" | |
| port = 5432 | |
| database = "prod_db" | |
| username = data.vault_kv_secret_v2.db_credentials.data["username"] | |
| password = data.vault_kv_secret_v2.db_credentials.data["password"] | |
| sslmode = "require" | |
| } | |
| resource "postgresql_role" "app_role" { | |
| name = "app_user" | |
| login = true | |
| password = "temporary_password_or_managed_separately" | |
| } | |
| variable "vault_address" { | |
| type = string | |
| description = "The URL of your Vault server" | |
| default = "https://vault.example.com:8200" | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| A | |
| amazon web service: aws |cloud, aws | |
| azure: az (local) | |
| B | |
| C | |
| cluster: cls | |
| D | |
| development: dev |environment | |
| E | |
| F | |
| G | |
| google cloud platform: gcp |cloud, gcp | |
| H | |
| I | |
| ingress: ing |cloud, gcp | |
| integration: int |environment | |
| internal ip: iip |gcp, network | |
| J | |
| K | |
| kubernetes: k8s (official), kub (local) |cloud, container, kubernetes | |
| L | |
| laboratory: lab |environment | |
| load balancer: lb |cloud, network | |
| M | |
| N | |
| O | |
| openShift: oc (local) |cloud, kubernetes | |
| P | |
| production: prd |environment | |
| pub/sub: psb |cloud, gcp | |
| public ip: pip |gcp, network | |
| Q | |
| queue: qu | | |
| R | |
| route: rut |network | |
| S | |
| T | |
| U | |
| V | |
| virtual machine: vm |cloud | |
| virtual ip: vip |network | |
| W | |
| X | |
| Y | |
| Z |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## show hidden/special characters | |
| :set listchars=eol:$,tab:>-,trail:~,extends:>,precedes:< | |
| :set list | |
| ## edit multiple file | |
| + split screen | |
| :vsp /PATH/TO/FILE # vertical split | |
| :sp /PATH/TO/FILE # horizontal split | |
| CTRL+W+ARROW # navigate to screen | |
| + add tab (buffer) | |
| :tabe /PATH/TO/FILE | |
| :bf # Go to first file | |
| :bl # Go to last file | |
| :bn # Go to next file | |
| :bp # Go to previous file | |
| :bw # Close file | |
| ## comment/un-comment mutiple lines | |
| + | |
| comment | |
| - visual block on needed lines: CTRL+V | |
| - insert mode on first line: SHIFT+I | |
| - add # | |
| - ECHAP | |
| un-comment | |
| - visual block on needed lines: CTRL+V | |
| - remove # on first line | |
| - ECHAP | |
| + | |
| comment: | |
| - shift-v | |
| - :norm i# | |
| uncomment: | |
| - shift-v | |
| - :norm x | |
| ## change colorscheme | |
| :colorscheme TAB | |
| ## if sudo forgotten | |
| :w !sudo tee % | |
| ## uppercase/lowercase | |
| ~ : case of current character | |
| guu : current line from upper to lower. | |
| gUU : current LINE from lower to upper. | |
| guw : end of current WORD from upper to lower. | |
| guaw : all of current WORD to lower. | |
| gUw : end of current WORD from lower to upper. | |
| gUaw : current WORD to upper. | |
| g~~ : Invert case to entire line | |
| g~w : Invert case to current WORD | |
| guG : lowercase until the end of document. | |
| gU) : until end of sentence to upper case | |
| gu} : end of paragraph to lower case | |
| gU5j : 5 lines below to upper case | |
| gu3k : 3 lines above to lower case | |
| --- VIMRC --- | |
| ## ident | |
| filetype plugin indent on | |
| " show existing tab with 4 spaces width | |
| set tabstop=4 | |
| " when indenting with '>', use 4 spaces width | |
| set shiftwid th=4 | |
| " On pressing tab, insert 4 spaces | |
| set expandt ab |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- OPENVPN | |
| # openvpn3 package | |
| - debian/ubuntu: openvpn3 | |
| # config management | |
| config list | |
| $ openvpn3 configs-list | |
| import a config profile: | |
| $ openvpn3 config-import --config /path/to/profile.ovpn --name MY-VPN --persistent | |
| manage/Modify an imported config | |
| $ openvpn3 config-manage --show --config MY-VPN | |
| remove an imported config | |
| $ openvpn3 config-remove --config MY-VPN | |
| manage Access Control Lists (ACL): | |
| $ openvpn3 config-acl --show --config MY-VPN | |
| $ openvpn3 config-acl --grant root --config MY-VPN | |
| # session management | |
| start a new VPN session | |
| $ openvpn3 session-start --config MY-VPN | |
| list all active VPN sessions | |
| $ openvpn3 sessions-list | |
| manage an active session | |
| $ openvpn3 session-manage --config MY-VPN --disconnect|--restart|--pause|--resume | |
| view real-time tunnel stats | |
| $ openvpn3 session-stats --config MY-VPN |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## Create user profile file | |
| Test if file exists | |
| $ Test-Path $profile | |
| False | |
| Create user profile file | |
| $ New-Item -Path $PROFILE -ItemType File -Force | |
| C:\Users\$USER\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1 | |
| ## Create Alias | |
| $ Get-Alias | |
| $ New-Alias ALIAS_NAME CMD_NAME | |
| or | |
| $ Set-Alias -Name ALIAS_NAME -Value 'CMD PARAM=VALUE' | |
| ## Create function | |
| Function _my_function { | |
| CMD PARAM=VALUE | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## add missing fonts | |
| Download zip -> https://github.com/BannedPatriot/ttf-wps-fonts | |
| $ sudo mkdir -p /usr/share/fonts/truetype/msttcorefonts | |
| $ cd ttf-wps-fonts && sudo ./install.sh | |
| $ sudo fc-cache -fv |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- VARIABLE | |
| export ID_USER | |
| export PASSWD_USER | |
| export MAIL | |
| export PROXY="proxy.domain.tld:PORT" | |
| export PROXY_SECURE="${ID_USER}:${PASSWD_USER}@${PROXY}" | |
| export http_proxy="http://${PROXY_SECURE}" | |
| export https_proxy="http://${PROXY_SECURE}" | |
| export no_proxy=localhost,.domain.tld,.other-domain.tld | |
| export http_auth_ntlm_domain="DOMAIN" | |
| --- ISSUE | |
| ## WSL Ubuntu Distro | How to solve `Operation not permitted` on cloning repository | |
| --> https://askubuntu.com/questions/1115564/wsl-ubuntu-distro-how-to-solve-operation-not-permitted-on-cloning-repository | |
| $ vim /etc/wsl.conf | |
| [automount] | |
| options = "metadata" | |
| $ wsl --shutdown | |
| ## if distro run with root as default user, back to normal user | |
| cmd: | |
| $ DISTRO_NAME.exe config --default-user USERNAME | |
| in distro: | |
| $ vim /etc/wsl.conf | |
| [user] | |
| default=USERNAME | |
| $ wsl --shutdown |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- STRUCTURE --- | |
| # items | |
| + basic | |
| --- | |
| key: value | |
| + list/sequence | |
| --- | |
| list: | |
| - item | |
| - item | |
| + dictionary | |
| --- | |
| item: | |
| key1: value | |
| key2: value | |
| + array | |
| --- | |
| array: [ 1, 2, 3, 4, 5 ] | |
| array: [ "one", "two", "three", "four" ] | |
| array: | |
| - 1 | |
| - "one" | |
| --- YALM QUERY --- | |
| https://github.com/mikefarah/yq | |
| https://mikefarah.gitbook.io/yq/usage/github-action | |
| https://www.baeldung.com/linux/yq-utility-processing-yaml | |
| # yaml structure | |
| key: "value" | |
| array: | |
| key1: "value1" | |
| key2: "value2" | |
| --> $ yq 'PARAMETERS' FILE.yaml | |
| # access properties | |
| + get value of key | |
| '.key' | |
| + all values of a node | |
| '.array[]' | |
| + search over all | |
| '.[] | select(. == "TEXT")' | |
| + search over the array with selector | |
| '.array[] | select(. == "TEXT*")' | |
| + recursive search over all | |
| '.. | select(. == "TEXT*")' | |
| + recursive search only on a node | |
| '.array | .. | select(. == "TEXT*")' | |
| # change/update properties | |
| + update values | |
| '.key = VALUE' | |
| '.array.key = VALUE' | |
| + add new nodes/properties | |
| -i '.key = VALUE' | |
| -i '.array.KEY = VALUE' | |
| + remove nodes/properties | |
| -i 'del(.KEY)' | |
| -i 'del(.array.KEY)' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ### YALM QUERY ### | |
| https://github.com/mikefarah/yq | |
| https://mikefarah.gitbook.io/yq/usage/github-action | |
| https://www.baeldung.com/linux/yq-utility-processing-yaml | |
| # yaml structure | |
| key: "value" | |
| array: | |
| key1: "value1" | |
| key2: "value2" | |
| --> $ yq 'PARAMETERS' FILE.yaml | |
| # access properties | |
| + get value of key | |
| '.key' | |
| + all values of a node | |
| '.array[]' | |
| + search over all | |
| '.[] | select(. == "TEXT")' | |
| + search over the array with selector | |
| '.array[] | select(. == "TEXT*")' | |
| + recursive search over all | |
| '.. | select(. == "TEXT*")' | |
| + recursive search only on a node | |
| '.array | .. | select(. == "TEXT*")' | |
| # change/update properties | |
| + update values | |
| '.key = VALUE' | |
| '.array.key = VALUE' | |
| + add new nodes/properties | |
| -i '.key = VALUE' | |
| -i '.array.KEY = VALUE' | |
| + remove nodes/properties | |
| -i 'del(.KEY)' | |
| -i 'del(.array.KEY)' |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment