We would love for you to contribute to {{template}} and help make it even better than it is today! As a contributor, here are the guidelines we would like you to follow:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Task locking with redis in celery is hard, and good examples are tough to come by. This is the approach that's | |
worked for me, based on great work that other folks have posted: | |
* https://breadcrumbscollector.tech/what-is-celery-beat-and-how-to-use-it-part-2-patterns-and-caveats/ | |
* http://loose-bits.com/2010/10/distributed-task-locking-in-celery.html | |
* https://redis.io/topics/distlock | |
This isn't polished,but hopefully it's useful. To verify it in our local test env we register the following test task in | |
our definitions file: | |
*** definitions.py *** |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# imports from matplotlib import pyplot as plt | |
from matplotlib import pyplot as plt | |
import pylab | |
from mpl_toolkits.mplot3d import Axes3D | |
from mpl_toolkits.mplot3d import proj3d | |
%matplotlib inline | |
%pylab inline | |
from sklearn.manifold import TSNE | |
from sklear.decomposition import PCA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FROM alpine:3.9 | |
ENV LIBRD_VER=1.3.0 | |
WORKDIR /tmp | |
#vishnus-MacBook-Pro:librd vrao$ docker images |grep lib | |
#lib proper_cleanup 675073279e9c 4 seconds ago 53.3MB | |
#lib cleanup 7456af7df73b 2 minutes ago 218MB | |
#lib simple 9724aed9519c 7 minutes ago 342MB |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
session = boto3.Session(profile_name='ariancho') | |
s3_client = session.client('s3') | |
display_name = s3_client.list_buckets()['Owner']['DisplayName'] | |
print(display_name) | |
for bucket in s3_client.list_buckets()['Buckets']: | |
print(s3_client.get_bucket_acl(Bucket=bucket['Name'])['Owner']['DisplayName']) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This gist describes the process of setting up direct grant access (oauth2 resource owner password flow) with keycloak and spring boot. We'll follow the | |
below steps - | |
1. Install keycloak - there are plenty of examples out there (even a docker image) | |
2. Create a demo realm and create a client within the demo realm with the settings as - | |
client-protocol: openid-connect, access-type: confidential, (implicit-flow+direct-access-grant+service-accounts)=enabled | |
3. Create 2 roles - developer and admin within the demo realm | |
4. Create 2 users - one with developer role and other with admin (Ensure that user is enabled, there are no "Required User Actions" and that the password has been reset (in the credentials tab) | |
Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups
command.
- List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe
Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.
- Reset the consumer offset for a topic (preview)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import hashlib | |
from functools import wraps | |
from django.core.cache import cache | |
from django.utils.encoding import force_text, force_bytes | |
def cache_memoize( | |
timeout, | |
prefix='', |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kubectl get pods | grep Evicted | awk '{print $1}' | xargs kubectl delete pod |
NewerOlder