Event Store with IDM local development

## Dev cheat sheet

### 1. Create Maxwell and Ankara users in the database:

```sql
CREATE USER 'maxwell'@'%' IDENTIFIED BY 'Passw0rd';
GRANT ALL ON maxwell.* TO 'maxwell'@'%';
GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'maxwell'@'%';

CREATE USER 'ankara'@'%' IDENTIFIED BY 'Passw0rd';
GRANT ALL ON ankara_instance.* TO 'ankara'@'%';
GRANT ALL ON ankara_scoring.* TO 'ankara'@'%';
GRANT ALL ON ankara_test.* TO 'ankara'@'%';
```

### 2. Configure MySQL (my.cnf) and restart the service

```txt
server_id=1
log-bin=master
binlog_format=row
```

### 3. Set up minikube for IDM

```shell
minikube start --memory 6144 --disk-size 40g --cpus 3
minikube addons enable ingress
minikube addons enable heapster
# following will ask for docker registry quay.io mkamornikov/password
minikube addons configure registry-creds
minikube addons enable registry-creds
minikube stop
VBoxManage modifyvm minikube --natdnshostresolver1 on
minikube start
kubectl create ns idm
```

### 4. While it's running, make sure to have the correct records in `/etc/hosts`:

```shell
minikube ip
# add following entry to /etc/hosts
<minikube ip> discovery sts signup console.sugar.multiverse login.sugar.multiverse smtpidpapi grafana hydra api 

# as we run event-store locally
127.0.0.1 event-store-001
```

### 5. Deploy IDM in minikube

```shell
make skaffold-idm-full-start
```

### 6. Create a teanant by opening http://signup in the browser:
   **Username:** [email protected]
   **Password:** $3nde7BACgz^!$xh
   **Tenant ID (autogenerated):** 1257765323

### 7. Login with scloud:

```shell
bazel run //projects/scloud -- auth login --disco=http://discovery --region=na --context=minikube
```

### 8. Export the event store service account secret for local usage:

```shell
kubectl -n idm get secret oauth-secret-event-store-001 -o json
base64 -d # decode the secret
cat > $HOME/oauth-secret-event-store.json # store it in a file
```

### 9. Or if deploying to `minikube`, copy the secret to the `evt` namespace:

```shell
kubectl -n idm get secret oauth-secret-event-store -o yaml | kubectl -n evt apply -f -
```

### 10. Install and configure AWS cli client

```shell
brew install awscli

# you can set keys in awscli configs
# ~/.aws/config, ~/.aws/credentials
# or export some ENV vars:
# AWS_SECRET_ACCESS_KEY=<your_secret_access_key>
# AWS_ACCESS_KEY_ID=<your_access_key_id>
# AWS_REGION=us-west-2
```

### 11. Authorize yourself

Usually the step above is enough but in case of enabled MFA we have to do some extra stuff.
In my case `sugararch` account was added to `base` group which forced MFA for ALL connections
including cli ones. The fix was to remove myself from that group.

As an alternative you can correctly authorize yourself from cli executing this scipt line by line
in each terminal used to run services which depend on AWS.
   
```shell
#!/usr/bin/env bash

# *IMPORTANT* this script need to be sourced, not executed
# remember to export AWS_PROFILE

MFA_ID=$(aws sts get-caller-identity  --query Arn --output text| sed 's,user/,mfa/,')
read -p "Type the MFA code for $MFA_ID: " MFA_CODE

response=$(aws sts get-session-token --serial-number "$MFA_ID" --token-code "$MFA_CODE")

export AWS_ACCESS_KEY_ID=$(echo "$response" | jq -r .Credentials.AccessKeyId)
export AWS_SECRET_ACCESS_KEY=$(echo "$response" | jq -r .Credentials.SecretAccessKey)
export AWS_SESSION_TOKEN=$(echo "$response" | jq -r .Credentials.SessionToken)

echo -n "Token expires at: "
echo "$response"| jq -r .Credentials.Expiration
```

### 12. Initialize the Event Store DB schema:

```shell
env AWS_SDK_LOAD_CONFIG=1 bazel run //projects/event-store/cli/server -- \
   migrate \
   --table=mkamornikov-evt-dev
```

### 13. Start a local Event Store instance:

```shell
env AWS_SDK_LOAD_CONFIG=1 bazel run //projects/event-store/cli/server -- \
    server \
    --secret=$HOME/oauth-secret-event-store.json \
    --table=mkamornikov-evt-dev \
    --grpc-addr=:30996
```

### 14. Create a new journal for Maxwell:

```shell
bazel run //projects/scloud event-store journal create maxwell
```

### 15. Start Mango ingestor:

```shell
# TODO: this service will need its own service account
env AWS_SDK_LOAD_CONFIG=1 bazel run //projects/event-store/cli/mango-ingestor -- \
   consumer \
   --disco=http://discovery \
   --region=na \
   --secret=$HOME/oauth-secret-event-store.json \
   --stream=mkamornikov-maxwell \
   --app=mango-ingestor \
   --table=mango-ingestor-chp-dev \
   --journal=maxwell \
   --instance-id=sugarinternal
```

### 16. Start Maxwell:

```shell
# !!! NEEDS TO HAVE A DATABASE FILTER TO PREVENT FEEDBACK FROM THE PROJECTION
docker run --rm -it \
    --env AWS_REGION=us-west-2 \
    --net=host \
    -v $HOME/.aws:/root/.aws zendesk/maxwell \
    sh -c 'cp /app/kinesis-producer-library.properties.example /app/kinesis-producer-library.properties && echo "Region=$AWS_REGION" >> /app/kinesis-producer-library.properties &&  echo "AggregationEnabled=false" >> /app/kinesis-producer-library.properties && bin/maxwell --user=maxwell --password=Passw0rd --host=localhost --metrics_type=http --filter="exclude: *.*, include: ankara_instance.*, exclude: *./.*_audit$/, exclude: *./.*cache.*/" --producer=kinesis --kinesis_stream=mkamornikov-maxwell'
```

In my case (`docker for mac`) i had to change `--host=localhost` to `--host=host.docker.internal`

### 17. Migrate the schema

```shell
bazel run //projects/ankara/projector -- \
    schema migrate \
    --db-host=localhost \
    --db-user=ankara \
    --db-password=Passw0rd \
    --db-name=ankara_scoring
```
    
### 18. Start the projector:

```shell
# TODO: this service will need its own service account
bazel run //projects/ankara/projector -- \
    project \
    --disco=http://discovery \
    --region=na \
    --secret=$HOME/oauth-secret-event-store.json \
    --journal=maxwell \
    --instance-id=sugarinternal \
    --db-host=localhost \
    --db-user=ankara \
    --db-password=Passw0rd \
    --db-name=ankara_scoring
```

### 19. Run integration tests:

```shell
bazel test --test_env=TEST_MYSQL_DSN="ankara:Passw0rd@tcp(localhost:3306)/ankara_test" \
    //projects/ankara/projector/pkg/projection/mysql:go_default_test
```