analogous-devAnalogous Dev Blog2025-02-22T13:09:51.108Zhttps://github.com/nuxt-community/feed-moduleThe analogous developer. Technology topics for newcomers and experts alike.analogous-dev.pngfavicon.icoCole Arendt 2022<![CDATA[Using the Kubernetes Python Client with AWS]]>using-the-kubernetes-python-client-with-aws2021-01-27T00:00:00.000Z
```python
import kubernetes
def k8s_api_client(endpoint: str, token: str, cafile: str) -> kubernetes.client.CoreV1Api:
kconfig = kubernetes.config.kube_config.Configuration(
host=endpoint,
api_key={'authorization': 'Bearer ' + token}
)
kconfig.ssl_ca_cert = cafile
kclient = kubernetes.client.ApiClient(configuration=kconfig)
return kubernetes.client.CoreV1Api(api_client=kclient)
```
There are other useful options in `kconfig`, like `kconfig.proxy` and
`kconfig.verify_ssl`. Have a look at the class for more details!
### Authenticate to AWS / EKS
Now we need to figure out how to get a bearer token to talk to EKS. I cannot use
the magic in `kubeconfig`, which led me to the [AWS CLI's `aws eks get-token`
command](https://docs.aws.amazon.com/cli/latest/reference/eks/get-token.html). I
chose to opt for a pure python solution in the
[`eks-token`](https://pypi.org/project/eks-token/) package. You can evaluate the
source for yourself [here](https://github.com/peak-ai/eks-token) and [more
specifically,
here](https://github.com/peak-ai/eks-token/blob/master/eks_token/logics.py).
With this module, we can acquire an EKS token easily:
```python
import eks_token
cluster_name = 'my-eks-cluster'
my_token = eks_token.get_token(cluster_name)
```
**NOTE:** It is worth checking whether the `boto3` or `aws` libraries provide
access to this functionality directly from Python as things improve!
### Now TLS
Unfortunately, the `kubernetes-python` client does not allow for inlining
the TLS CA Certificate. As a result, we have to write it to a temp file
(which by inspection is exactly what the `kubeconfig` approach is doing)
```python
import boto3
import tempfile
import base64
def _write_cafile(data: str) -> tempfile.NamedTemporaryFile:
# protect yourself from automatic deletion
cafile = tempfile.NamedTemporaryFile(delete=False)
cadata_b64 = data
cadata = base64.b64decode(cadata_b64)
cafile.write(cadata)
cafile.flush()
return cafile
bclient = boto3.client('eks')
cluster_data = bclient.describe_cluster(name=cluster_name)['cluster']
my_cafile = _write_cafile(cluster_data['certificateAuthority']['data'])
```
### Put it all together
Armed with a bearer token and transport layer security (TLS), we now have the
tools we need to succeed!!
```python
api_client = k8s_api_client(
endpoint=cluster_data['endpoint'],
token=my_token['status']['token'],
cafile=my_cafile.name
)
api_client.list_namespace()
```
## Go wild
The first time `list_namespace()` returned data, I was ecstatic.
Authentication successful! Now the world is your oyster.
All that's left is perusing the [API client
docs](https://github.com/kubernetes-client/python/blob/master/kubernetes/README.md)
and refreshing AWS creds every so often!
For example, creating a configmap:
```python
my_configmap = kubernetes.client.V1ConfigMap(
api_version='v1',
metadata={'name': 'my-configmap'},
kind='ConfigMap',
data={'my_file.txt': 'mycontent'}
)
api_client.create_namespaced_config_map(namespace='default', body=my_configmap)
# NOTE: to update a configmap, you need to
# use k8s_client.replace_namespaced_config_map
#
# If it already exists, create will give a 409 conflict
```
Have fun!!
[The source for this article is available here (along with a few extra
examples)](https://github.com/colearendt/example-python-kubernetes)
## Outtakes
Yes, there were some outtakes:
- Standing up [`mitmproxy`](https://mitmproxy.org/) in a docker container and
setting `kconfig.proxy='http://localhost:8080` and `kconfig.verify_ssl=False`
before initializing `api_client` so I could see the requests being made to the
Kubernetes cluster and verify whether authentication was being sent
- Flailing on temp files (today, I learned that python disposes of temp files
rather quickly by default)
- Trolling the internet and finding an unfortunate absence of docs on this
topic. "Is this obvious to everyone else? Am I doing something wrong?" Classic
questions of a lonely explorer.
- Accidentally installing the `aws` package into my `pyenv` and breaking my
terminal's ability to assume roles when in certain directories.
Programming can be hard! It is so nice to know we are not alone!
]]><![CDATA[Using sssd in a Playground Without TLS]]>sssd-without-tls2022-10-01T00:00:00.000Z NOTE: Do not use this setting in any "real" environment with "real" users, passwords, sensitive data, etc.
## Give it a Shot
### Create Users
First, we need to create and populate our LDAP server. Let's go ahead and do that. It is easiest if we create a file
with users first. For a more advanced LDIF file, check
out [the repository associated with this post](https://github.com/colearendt/container-playground):
_users.ldif_
```ldif
version: 1
## Entry 1: dc=angl,dc=dev
#dn: dc=angl,dc=dev
#dc: angl
#o: Angl Dev
#objectclass: top
#objectclass: dcObject
#objectclass: organization
#
## Entry 2: cn=admin,dc=angl,dc=dev
#dn: cn=admin,dc=angl,dc=dev
#cn: admin
#description: LDAP administrator
#objectclass: simpleSecurityObject
#objectclass: organizationalRole
#userpassword: {SSHA}+FquX8RcwTtBPo7mu2pgSvjaQYX9HpCL
#
#
# Entry 3: cn=engineering_group,dc=angl,dc=dev
dn: cn=engineering_group,dc=angl,dc=dev
cn: engineering_group
gidnumber: 500
memberuid: joe
memberuid: julie
objectclass: posixGroup
objectclass: top
# Entry 4: dc=engineering,dc=angl,dc=dev
dn: dc=engineering,dc=angl,dc=dev
dc: engineering
description: The Engineering Department
o: Engineering
objectclass: dcObject
objectclass: organization
objectclass: top
# Entry 5: cn=joe,dc=engineering,dc=angl,dc=dev
dn: cn=joe,dc=engineering,dc=angl,dc=dev
cn: joe
gidnumber: 500
givenname: Joe
homedirectory: /home/joe
loginshell: /bin/sh
mail: joe@angl.dev
objectclass: inetOrgPerson
objectclass: posixAccount
objectclass: top
sn: Golly
uid: joe
uidnumber: 1000
userpassword: {MD5}j/MkifkvM0FmlL6P3C1MIg==
# Entry 9: cn=julie,dc=engineering,dc=angl,dc=dev
dn: cn=julie,dc=engineering,dc=angl,dc=dev
cn: julie
gidnumber: 500
givenname: Julie
homedirectory: /home/julie
loginshell: /bin/sh
mail: julie@angl.dev
objectclass: inetOrgPerson
objectclass: posixAccount
objectclass: top
sn: Jolly
uid: julie
uidnumber: 1001
userpassword: {MD5}FvEvXoN54ivpleUF6/wbhA==
```
You will notice that the first two entries are commented out. They are included to represent a _complete_ LDIF file.
However, the `osixia/docker-openldap` container help us by provisioning these automatically.
Further, you will notice that passwords are included. This makes things easier for our playground, but is _definitely_ a
bad idea in real life / production applications.
### Create LDAP Server
Now let's create the server itself!
```bash
docker network create playground-network
docker run \
-d --name openldap --rm \
-p 389:389 \
--network playground-network \
-v $(pwd)/users.ldif:/container/service/slapd/assets/config/bootstrap/ldif/50-bootstrap.ldif \
-e LDAP_TLS=false \
-e LDAP_DOMAIN="angl.dev" \
-e LDAP_ADMIN_PASSWORD="admin" \
osixia/openldap:1.5.0 \
--copy-service --loglevel debug
```
And check that it is working
```bash
docker exec -it openldap ldapsearch -D cn=admin,dc=angl,dc=dev -b dc=angl,dc=dev -w admin cn
docker exec -it openldap ldapsearch -D cn=admin,dc=angl,dc=dev -b dc=angl,dc=dev -w admin cn=julie \*
```
If you look carefully, you will notice that:
1. We created a persistent network for our containers to share
2. We provisioned users from our `ldif` file
3. We disabled TLS on the service
4. We bumped up the logging verbosity for debugging purposes
These are all useful tidbits to dig into if you are not familiar!
### Configure sssd Server
It is possible to run `sssd` in a fairly vanilla `ubuntu:jammy` container.
```bash
docker run -it --name sssd --rm --network playground-network ubuntu:jammy bash
apt update && apt install -y sssd ldap-utils vim
```
Then you need to create your `sssd.conf` file. Notice our magic
option `ldap_auth_disable_tls_never_use_in_production=true`. This will be the magic that makes things work for us!
```bash
cat << EOF > /etc/sssd/sssd.conf
[sssd]
config_file_version = 2
services = nss, pam
domains = LDAP
[nss]
filter_users = root,named,avahi,haldaemon,dbus,radiusd,news,nscd
filter_groups =
[pam]
[domain/LDAP]
id_provider = ldap
auth_provider = ldap
chpass_provider = ldap
sudo_provider = ldap
enumerate = true
# ignore_group_members = true
cache_credentials = false
ldap_schema = rfc2307
ldap_uri = ldap://openldap:389
ldap_search_base = dc=angl,dc=dev
ldap_user_search_base = dc=angl,dc=dev
ldap_user_object_class = posixAccount
ldap_user_name = uid
ldap_group_search_base = dc=angl,dc=dev
ldap_group_object_class = posixGroup
ldap_group_name = cn
ldap_id_use_start_tls = false
ldap_tls_reqcert = never
ldap_tls_cacert = /etc/ssl/certs/ca-certificates.crt
ldap_default_bind_dn = cn=admin,dc=angl,dc=dev
ldap_default_authtok = admin
access_provider = ldap
ldap_access_filter = (objectClass=posixAccount)
min_id = 1
max_id = 0
ldap_user_uuid = entryUUID
ldap_user_shell = loginShell
ldap_user_home_directory = homeDirectory
ldap_user_uid_number = uidNumber
ldap_user_gid_number = gidNumber
ldap_group_gid_number = gidNumber
ldap_group_uuid = entryUUID
ldap_group_member = memberUid
ldap_auth_disable_tls_never_use_in_production = true
use_fully_qualified_names = false
ldap_access_order = filter
debug_level=6
EOF
chmod 600 /etc/sssd/sssd.conf
```
Now let's start the `sssd` service
```bash
sssd -i
# should see some log messages that suggest things are happening!
```
### Be sure it works!
Now let's make sure that this works by starting another shell in our `jammy` container.
```bash
docker exec -it sssd bash
id joe
# uid=1000(joe) gid=500(engineering_group) groups=500(engineering_group)
id julie
# uid=1001(julie) gid=500(engineering_group) groups=500(engineering_group)
```
## Using `docker-compose`
For playground environments like this, `docker-compose` makes this setup much easier to architect and reuse. You can
use [my example compose setup](https://github.com/colearendt/container-playground) if you prefer.
```bash
cd compose/
docker network create playground-network
NETWORK=playground-network docker-compose -f ldap.yml -f sssd.yml -f network.yml up -d
docker exec -it compose_sssd_1 bash
sssd -i >/tmp/sssd.log 2>&1 &
id joe
```
## Review
Well done! You have successfully started your own `sssd` container. Although this is very much a toy, it is a
great "jumping off point" to learn and understand how `sssd` works in more detail!
Any time you need a toy LDAP server for `sssd`, just remember: `ldap_auth_disable_tls_never_use_in_production = true`.
]]><![CDATA[Helm Intro and Helm Cheatsheet]]>helm-cheatsheet2022-10-10T00:00:00.000Z
## What's Next?
The [helm project](https://helm.sh/) is an open source project. There is much
that could be improved, and many applications that need helm charts or need
improved helm charts. You can make a difference! If you are interested in
learning more, check out the [helm tag](/?tag=Helm) on this blog to see other writing on
the topic, and start poking around on [ArtifactHub](https://artifacthub.io/),
where lots of charts are centralized for easier searching!
]]><![CDATA[Websockets and Cloudflare Workers]]>websockets-and-cloudflare-workers2025-02-22T00:00:00.000Z {
console.log(event);
server.send("Hello from the other side!");
});
server.addEventListener("close", () => {
console.log("WebSocket closed");
});
return new Response(null, {
status: 101,
webSocket: client,
});
}
}
```
And integrate into the `index.ts` file:
_index.ts_
```
import { WebSocket } from "./endpoints/websocket";
...
openapi.all("/api/ws", WebSocket);
```
### Test it!
Now how do you test websockets!? Well there are some cool tools out there on the web.
But I just wanted to test locally, so I made a little python script to test with:
_wstest.py_
```
import argparse
from websocket import create_connection
def main():
parser = argparse.ArgumentParser(description='WebSocket test client')
parser.add_argument('--domain', '-d', nargs='?', default='localhost:8787', help='The hostname to connect to (e.g. localhost:8787)')
parser.add_argument('--tls', '-t', action='store_true', help='Use TLS')
parser.add_argument('--path', '-p', nargs='?', default='/api/ws', help='The path to connect to (e.g. /api/ws)')
args = parser.parse_args()
if args.tls:
protocol = "wss"
else:
protocol = "ws"
# ensure that args.domain does not have a protocol prefix
# parse domain as a url
domain = args.domain.split('://')[1]
path = args.path
# ensure that there is only one slash between domain and path
if path.startswith('/'):
path = path[1:]
if domain.endswith('/'):
domain = domain[:-1]
ws_url = f"{protocol}://{domain}/{path}"
ws = create_connection(ws_url)
print(f"--> Connected to {ws_url}")
print("--> Sending 'Hello, World'...")
ws.send("Hello, World")
result = ws.recv()
print(f"<-- Received '{result}'")
try:
while True:
msg = input('--> ')
ws.send(msg)
result = ws.recv()
print(f"<-- Received '{result}'")
except KeyboardInterrupt:
print("\nClosing connection...")
finally:
ws.close()
if __name__ == '__main__':
main()
```
You will need one little library:
```
pip install websocket-client
```
And then you should be good to test!
```
# use the port that your websocket worker is running on from `wrangler dev`
python3 ./wstest.py -d localhost:59462
```
## Deploy to Cloudflare
This should be pretty straightforward! Deploy your worker!
```
wrangler deploy
```
Then redirect the websocket test script and see what happens! (Make sure to include the TLS flag!):
```
python3 ./wstest.py -d my-deployed-domain.workers.dev -t
```
## More debugging
There are lots of snags you can hit with websockets, especially if you start using the browser.
- Beware of CORS
- Be mindful of `ws://` vs. `wss://` (for TLS connections) protocol issues
- Be careful of client code, which needs to handle reconnections and can cause problems disconnecting unintentionally
- Firewalls, enterprise security restrictions, and reverse proxies can cause all sorts of weird issues with websockets!
- It is probably worth using socket.js or having some fallback mechanisms handled... but more on that another time.
## Successful Websocketing!
Congratulations! You have hopefully deployed a websocket server successfully to Cloudflare!
I would love to hear how it went, if you ran into issues, how this article can be improved, or what you ended up building! [Let me know!](mailto:info@analogous.dev)
Thanks!
]]>