Developer Journal - March 2019

This developer-journal entry was generated with a local AI model from my public GitHub activity, then reviewed before publication.

Digest

March 2019 focused on Docker configuration updates, Airflow version management, and Kubernetes script refinements in docker-airflow, alongside Dockerfile adjustments and dataset processing enhancements in mrsharky.

Docker Apache Airflow

Updated Airflow version in Dockerfile (commit 722ad9d), refined Dockerfile for Celery executor (commits 001eacee, e48390d), and adjusted Kubernetes scripts for variable connections and timeouts (commit fca0623d).

Performed rebasing of development branches (PR #2) and merged incremental fixes for Dockerfile and Kubernetes scripts (PRs #3–#5).

Notable sources

• https://github.com/dafrenchyman/docker-airflow/commit/722ad9d0bc7e34ec5b97cf7c0c96f76b7795e3f4
• https://github.com/dafrenchyman/docker-airflow/commit/fca0623d1019df3602090520134295326c076e3a

main java codebase

Modified Docker scripts for database handling (commit 3baac5a) and updated Dockerfile to build Java libraries (commit a8c451f).

Added dataset processing examples including SQL scripts and a NOAA temperature dataset (commit 997862e).

Notable sources

• https://github.com/dafrenchyman/mrsharky/commit/997862ec15d8738cd2f85d054b4727a19eb8ef47

Helpful Java Spark stuff

Added RowToDouble utility class for Spark processing (commit 1c1ae38).

Notable sources

• https://github.com/dafrenchyman/spark/commit/1c1ae3833f0aae1a939dbd96e3426e33edb81e36

Sources

• dafrenchyman/docker-airflow
• dafrenchyman/mrsharky
• dafrenchyman/spark
• 722ad9d
• 13c98f3
• ca3f711
• e250712
• cf8c7b3
• f5d7a56
• e48390d
• 001eace
• 0c9c3dd
• fca0623
• PR #2: Rebase test
• PR #3: JP - Playing with dockerfile
• PR #4: JP - Celery fix
• PR #5: JP - Fixed the DockerFile; updated the variable connections; switch…
• 3baac5a
• a8c451f
• 997862e
• 1c1ae38