SCRUM ceremonies get a bad rap sometimes—too many meetings, not enough keyboards. But when they’re done right, they’re some of the most engineer-friendly rituals in modern software development. Each ceremony exists for a reason, and collectively they give developers clarity, autonomy, and a steady feedback loop that makes building software better.

Here’s why engineers actually love them.

Sprint Planning: Clarity Without Guesswork

Purpose:
Sprint Planning aligns the team on what will be built in the upcoming sprint and why it matters. The team reviews priorities, estimates effort, and commits to a realistic scope.

What engineers get out of it:
No more vague expectations or surprise deadlines. Engineers get a clear, agreed-upon plan that respects capacity and technical complexity. It’s also the moment where devs influence scope, push back on risky assumptions, and surface dependencies early. The result? Fewer fire drills and more focused coding time.

Daily Standup: Fast Alignment, Zero Drama

Purpose:
Daily Standup keeps the team synchronized by answering three simple questions: what I did, what I’ll do, and what’s blocking me.

What engineers get out of it:
Standups prevent small issues from becoming big ones. Blocked? You get help fast. Finished early? You can pivot quickly. It’s a lightweight way to stay aligned without long status emails or meetings. When done right, it’s 10–15 minutes that saves hours of rework.

Backlog Refinement: Fewer Surprises, Better Builds

Purpose:
Backlog Refinement ensures upcoming work is well-defined, sized, and technically feasible before it ever hits a sprint.

What engineers get out of it:
This is where ambiguity goes to die. Engineers can ask hard questions, suggest better approaches, and flag tech debt before it becomes a sprint problem. Cleaner tickets mean smoother implementation—and fewer “wait, what does this mean?” moments mid-sprint.

Sprint Review: Your Work, Visible and Valued

Purpose:
Sprint Review showcases completed work to stakeholders and gathers feedback on what was built.

What engineers get out of it:
Engineers get to show real, working software—not slides. It creates visibility into the impact of their work and tightens the feedback loop with users and stakeholders. That feedback often leads to better designs and fewer wrong turns in future sprints.

Retrospective: Continuous Improvement, Engineer-Driven

Purpose:
The Retrospective reflects on what went well, what didn’t, and what the team can improve next sprint.

What engineers get out of it:
This is the team’s chance to fix the system, not just the code. Pain points get named. Processes improve. Small changes compound over time. For engineers, retros are empowering—they turn frustration into action and give teams ownership over how they work.

The Big Picture

SCRUM ceremonies, particularly when led by engaged engineers themselves, are the most important meetings in software development.
They reduce frustration, amplify engineer’s voice, and force clarity. It’s the best place to improve your craft.

Know the agenda. Leave when the agenda is met, and you won’t waste a minute of your week.

When you’re optimizing a data lake, your only metrics are cost and time. You’re working with
expensive clusters of worker machines, and can approximate cost as:

1

COST = NUMBER_OF_WORKERS * COST_OF_WORKERS * TIME

Memory/CPU usage, spill, and shuffle can be symptoms for diagnosing problems. Ultimately, the real objective is ensuring the job completes in an appropriate amount of time and for a reasonable cost.

Spill

First, consider spill. Spill means you ran out of memory, are writing to disk, and will have to be read back again.
Disk access is slow and costly.

Shuffle

Next, think about shuffle. Shuffle is an undesirable situation where data is on the wrong machine and
must be reorganized on workers, adding significant network time. This is not necessarily a problem, but a symptom to be avoided.

Resources (CPU/Memory)

Memory and CPU usage should be monitored, but the focus is efficient completion.
The ideal scenario is that “100% of compute” is used and memory is “100%” utilized, ensuring efficient use of resources.
Less vCPU or Memory will always save cost, as long as they are not adding significant time.

Tune your cluster with common sense. If you expect a memory-intensive task, add memory-weighted machines; otherwise, use compute-heavy machines to crunch through.

Clusters have a min-max number of workers that can manage cost and are how you scale. Well planned
jobs can scale by expanding the number of workers. Idle machines are always a wasted expense.
Common traps include relying on single-machine libraries (like python pandas) or writing code outside of spark frameworks that can’t be shared across cluster workers.
Bigger isn’t always faster and underutilizing specialized hardware (high-memory or GPUs) comes with big price tags.
Ensure you use frameworks like Spark, distributed data frames, or SQL to allow workloads to distribute across machines.
Effectively using available resources and adjusting the number of workers allows you to keep processes scalable and cost-efficient.

It’s expected that clusters are temporary. They spin up for a job and turn them off as soon as possible.
Running them 24 hours a day for small, frequent tasks is an expensive proposition in a data lake.
Additionally, after jobs are done, there is a period of idle before the machines are shut off.
A low idle shutoff setting will ensure clusters turn off quickly when they’re done.
This loses cached memory, but brings quick savings.

You’re choosing a power supply for a device, like a retro console. The device has defined requirements, often noted on the device by the plug.

These requirements include: voltage, amperage, the plug’s size (see barrel plugs) AND the “polarity” should be checked in case it’s reversed (rare.)

Electrical Requirements

In short, your power supply must match the Voltage and provide AT LEAST the amount of Amperage.

1
2
3
4

NES     9V    1.3A
SNES    10V   850mA
Genesis 10V   800mA
Sega CD 9V    1200mA

1000mA = 1A

Polarity

As a convention, the inside of barrel plugs is usually positive. This should be checked before using a replacement power supply. These markings on the device describe polarity. The left is most electronics.

Where I lose you

You’re choosing a Power Regulator (a.k.a. Voltage Regulator) for one or many electronics. These devices’ stated purpose is to provide a consistent and clean voltage.

You can plug it into your SNES and as long as your Power Regulator maintains 10 volts (+/- a bit) your Nintendo will be happy. An undamaged SNES will draw between 0, and 850mA.

You require a 10V power supply of 850mA or higher.

You can plug multiple devices into a power supply, like your PC does, and sum the Amperage requirements. SNES and Genesis coincidently need 10V, so a 10V power supply is required.

It’s pretty hard to be clever if you’re an executable file. Understanding how your programming interpreter views the world can save you days of development time and release you from the perils of virtual environments.

After you hand your interpreter code, you will likely be including libraries. The PATH is what will be used to find available modules and code.

Let’s check it out in Python.

1
2

> \>> python -c "import sys; print(sys.path)"
['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/home/trevor/.local/lib/python3.10/site-packages', '/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.10/dist-packages']

Some quick things to notice.

• Python will crack open zip files and use them just like they are directories. Awesome.
• /usr/lib/python3.10... python ships with some core libraries, and those get stored in a shared part of the file system, available to more than one user.
• /home/trevor... libraries get installed in your user’s home dir – best choice yet.

Now let’s go custom.

1

PYTHONPATH=.:.venv

I like to live dangerously and include the present working directory, ., and if there is something in the folder named .venv it’s always something I will want to include. venv works too, but conflicts with python’s venv package setup.

Between these directories and my environment variables I can completely control the state of my environment. If I want to install a lib for all of my apps, go for it: python -m pip install {lib} – I never want to do that. Maybe I’ll do it begrudgingly if there is a command line executable I want available.

When I want to install a module for one project (which I always do, hard disk is cheap), I install to .venv.

For pip this is the -t argument.

1

python -m pip install -t .venv {lib}

And that’s pretty much my virtual environment. You don’t have to activate, or deactivate anything. There is no state being changed in my environment to confuse me. Every time I run python everything works exactly as I expect it to. If something was misbehaving I have full competence over my environment and would be able to debug it quickly.

Is it inconvenient to remember the -t flag, maybe. I place a Makefile in my project and I haven’t typed a pip install since. The Makefile I use is idempotent as well, so I can test, run, or serve and it will only install or update packages once and exactly when it needs to.

This doesn’t solve running multiple versions of your interpreter. I download mine individually at this point. It’s pretty hard to get organizations to keep bleeding-edge on interpreters, so this hasn’t been a problem for me. Once again, the binary I’m using gets checked into the Makefile. If I’m testing cross binaries, I use a framework, like tox. If I’m running something, I like being explicit with what I’m using.

Putting a pin in the text editor holy war – do you use VIM, or Emacs?! We may point out hipster tendencies by using these “way better” vintage technologies, but there is still extreme value here. What you’re missing out on is having scripting in your editor. This will add years to your life. If you ever had the need to bulk edit files in your repository, or felt monotony while making a code update, read on. I choose VIM and I’m going to walk you through one catastrophy of a development day. If you can do this in your editor, you keep using that. This will presume you can reasonably navigate VIM, but should show the potential of using any of these technologies.

So let’s say you find yourself in a project with 128 YAML files that all need the same update. Do you manually edit each file, with your mouse? Brutal. Do you write a python script to parse every YAML file, edit it and write it back in place? Savage. This certainly is some project that predates you. People have probably left the company because of these 128 files, and now you need to update them? Enter Editor Scripting.

Case Study

Edit 128 YAML files without losing your mind. Remove the bad_values key and all it’s child values.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

file: 1
bad_values:                           #
  value: "1"       # delete these lines in many files.
  author: last guy who worked here    #
rest:
  of:
  - file
bad_values1:                           #
  value: "1"       # delete these lines in many files.
  author: last guy who worked here    #
bad_values2:                           #
  value: "2"       # delete these lines in many files.
  author: last guy who worked here    #
bad_values3:                           #
  value: "3"       # delete these lines in many files.
  author: last guy who worked here    #

1. Target the files you need to edit.

Two strong choices here – either aggregate a list of filenames (which will be super easy if you know how to do visual editing in VIM):

1

vim file1.yml file2.yml ...

or throw a regex at it. grep? Yes, please.

Open vim in folder of your choosing.

1

:grep -R bad_values *

Your screen will probably throw up on you for a spell, and then you’ll press enter a couple times and find yourself on the first occurance of bad_values. Completely unobvious to you, you can type :cnext<enter> to move to the next occurance.

2. Make the edit.

Think of this exercise as a script. You don’t want to move over into edit mode and stumble around the text. You wish to tell the machine what you would like it to change.

In this case you know you will be on the bad_values line. You want to delete the present line, and the two lines that follow it. You just happen to solve this 3 line delete situation by pressing 3dd. Generally speaking, VIM operates by receiving commands as a “motion” and an “action.” A motion could be “the next word”, “the next 3 sentences”, or read until you see a “,”. If you can think it up VIM can do it. Actions include: delete, replace, substitute, add one to the number. You’ll need to learn a handful of each of these, and you’ll spend the reset of your career remember-forgetting them. Know a couple well and move on.

Now you’re getting somewhere. You can make a todo list of files or lines, and run a single command on them. You don’t have to write 10’s of lines of code to read in files, modify using your clever technique, and then write. This is usable. But this probably seems a bit basic to you.

3. Multi-step Edits (recording)

Realistically you probably aren’t going to have as much luck as above at some point. You’ll learn VIM well enough to know you can make a couple deterministic motions to get into the right place, but you’ll be in the same situation as before (but looking cooler while doing it.) In these situations you can tell VIM to record your keystrokes and replay them for you.

For instance in the YAML example, maybe I find myself in the file, I wish to update all the bad_values value keys. Let’s get ridiculous. Follow along by pressing the code keys below, do not add or escape any additional keys.

1. Start a new recording q, name it something, let’s say w, but you have 26 letters to choose here.
2. Search for the areas to update. / (search) for bad_values. This highlights your targets and gets you most of the way there.
3. You really want to change the line below your search target, so let’s press the down key (j if you’re a pro.)
4. Let’s say we want to increment the number in the quotes. <ctrl>+a. If you want to add 10, 10<ctrl>+a. want to update it because it’s a string? ci" will “change” “inside” the ‘“‘ quotes. No kidding.
5. Stop the recording. q
6. Repeat this process as many times as you like @w.

Conclusion

I will never tell a developer to use any certain tool, but having at least one scripting editor in your tool belt will take you to the next level.

Kubernetes has been a large force in the app deployment industry for some time now, and this article is by no means the authority on Kubernetes. I’ve personally have had some stop and go with adoption, and knowing a technology “enough” can lead to bad practices. This article will walk you through the basic logistics of developing for K8s on your machine without “experimenting in production.” The goal of this article is to note commands for new k8s developers and to demonstrate validating k8s manifests locally in order to cut down development time.

We’ll create a Kubernetes Cluster locally on our machines which will let us test our code more quickly, and give us a space to experiment and make bad decisions.

Creating a Local Kubernetes Cluster

We’ll use minikube to develop on Macintosh. MicroK8s may be a good option for linux. In both cases we’ll need to have docker installed as well.

• Install Docker
• Install minikube
• Install kubectl - brew install kubectl

If you’ve installed everything correctly you can run:

1

minikube start

Perhaps you’ll note your laptop’s fan trying to take off while you’re regaled with emojis. Hopefully we’ll see the following surfer:

1

🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

This is important to note. kubectl is the command we’ll be doing work with, and they were thoughtful enough to reconfigure our environment to point to the minikube cluster to get started. You may eventually have many clusters you’ll want to connect to (~/.kube/config is a good place to save them) and you can point kubectl to different clusters using environment variables.

Connecting to minikube

You have your own kubernetes cluster! You’re free to use this as you will. kubectl cluster-info is a good sanity check.
You’ll probably use the following commands useful for running kubernetes manifests you’re working on.

1
2
3
4
5
6
7
8
9
10
11

# deploy your deployment/service/pod...
kubectl apply -f path/[file.yml]

# undo
kubectl delete -f path/[file.yml]

# or if you've made a mess, and just want to kill stuff.
kubectl delete [pod|deployment] name-to-delete

# process list
kubectl get [pods|deployment]

If you want to check the syntax of manifest files you’ve written, consider the following. Checking your syntax on a local cluster without bumping into all your coworkers is already a big win. You’ll have reliable quick results without having to a server.

1
2

# this checks the `manifests` folder
kubectl apply  -f manifests --dry-run

docker images in minikube

You’re eventually going to want to use a docker image you have built. Unfortunately minikube is running in it’s own container and your local machine’s images won’t be available there. It will probably make sense to share a docker registry between the two hosts, but a quick solution is to just build the docker images where you want them (in minikube.) Your code and local files should all show up in the minikube container.

You can change your shell’s environment to point to the minikube docker:

1
2
3

eval $(minikube docker-env)

docker build . -t your-tag-name # this will now build to your minikube repo

NOTE: You can always ssh into the minikube instance with minikube ssh if you want to tinker.

image pull problems

If you’re having problems with pulling docker images and you already have them locally, you can use the imagePullPolicy configuration as IfNotPresent or Never.

Otherwise updating .docker/config.json with your private registry may help.

connecting to your services

minikube is going to require an extra step to provide service access to your host machine. AKA: If you want to connect to your web service from your browser, you’ll have to do this.
If you have set up a k8s service, you can run minikube service {service-name} and a tunneling process will start up (and maybe open your browser!)

If you are using LoadBalancer, minikube tunnel may work better for you.

conclusion

As technologists, we always need to be sharpening our tools. Hopefully the steps above have helped you improve your development environment and sped up your kubernetes development process.

MoSCoW method

Feature creep is a real thing. It’s pretty hard to present any technology proposal
to a group of engaged engineers without large “blue sky” converstations starting.

The MoSCoW method gives you four one word classifications that will make your intentions clear.

• MUST - we will build this now
• SHOULD - not committed to
• COULD - not planning on it
• WON'T - not doing

The wiki page is so good that I’m just going to link to the MoSCoW method.

Data Scientists breaking up their notebooks into standard methods (or modules) can improve their process without interrupting their workflow.

The following proposed organization of code would aid in:

• faster deploy times
• less code refactoring
• interchangable functions
• easier code hand offs
• reproducable results

How do you run this? Try the makeup (pypi.org) framework designed just for this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

"""
Code taken from: https://scikit-learn.org/stable/tutorial/statistical_inference/supervised_learning.html

It has been modified to encapsulate and label sections of code.
"""
import numpy as np
from sklearn import datasets
from sklearn.neighbors import KNeighborsClassifier


def load(**kwargs):
    """
    "Get the data" this sparse function allows a framework
    to be able to swap out different data sets easily.
    """
    iris_X, iris_y = datasets.load_iris(return_X_y=True)
    return iris_X, iris_y


def features(X, y, **kwargs):
    """Optional method to transform data, add or remove columns"""
    # if i had pandas here, I would do something!
    return X, y


def split(X, y, **kwargs):
    np.random.seed(0)
    indices = np.random.permutation(len(X))
    X_train = X[indices[:-10]]
    y_train = y[indices[:-10]]
    X_test = X[indices[-10:]]
    y_test = y[indices[-10:]]

    return (X_train, y_train),\
           (X_test, y_test)


def train(X, y, **kwargs):
    """Create and fit a nearest-neighbor classifier"""
    model = KNeighborsClassifier()
    model.fit(X, y)

    return model


def predict(model, X, **kwargs):
    return model.predict(X)

tg.

Environment Variables or “ENV vars” are under used and often misunderstood.
They are critical for separating configuration from any code you write. By
learning how to use this SHELL resource in your programs your code will
become more extensible and useful to a larger group of people and environments.

What is an ENV Var?

Environment Variables are variables for your Terminal Shell. They are passed to every
program you run and can be used by those programs. If you run commands in your IDE,
these are still in play (there is probably a field that you could define these in.)

Do I have ENV Vars? Yes. From a terminal, type:

1

env

That’s all of them! Ignore some of the gobblety-gook. Get these right and your
work environment will “just work” and feel magical. Some worth pointing out:

• USER: Want to default to using your username in your code? How about defaulting to the username of whomever is using the code? Use this variable.
• HOME: How many times do you have /Users/{your_username}/... pasted in your code? Use this variable. Or maybe you want to use…
• PWD: Your Present Working Directory.
• EDITOR: Choose what editor application will open when you need to edit a file. (Think git asking for commit message.) vim, nano are examples, but there is no reason you can’t use Sublime or any IDE!
• PATH: A : separated list of where your terminal looks for programs to run. FUN!
• PS1: Change the prompt in your terminal.

Using ENV Vars

ENV vars do everything you expect a variable to do in any programming environment. See this example in bash, and most shells.

1
2
3
4
5
6
7
8
9

# set it
export MYVAR=123

# read it
echo $MYVAR

# remove it!
unset MYVAR

These values are useful if you’re doing regular tasks in your terminal…

1

curl $MY_FREQUENTLY_USED_SERVICE

…and they also are a great way to inject configuration into your programs! After you “export” your variable 100% of the programs you run in that terminal will get the variable passed to it. Most programs may ignore it, but your program won’t

In python, as an example you can retrieve env vars with:

1
2
3
4
5
6

from os import environ

required_var = environ['REQUIRED_VAR']  # will throw a `KeyError` if doesn't exit.

SOME_DEFAULT_VALUE = 456
myvar = environ.get("MYVAR", SOME_DEFAULT_VALUE)

SOME_DEFAULT_VALUE is optional, it could just be excluded, but if you can do something useful with your program without it defined consider adding a useful default value. Maybe you need a folder to save data into, no good ideas to put it? $HOME may make sense.

This will look for a DATA_DIR variable that they could have set, or it will default reasonably to a directory called data in their home directory.

1
2
3
4

from os import environ

DEFAULT_DIR = environ.get("HOME") + "/data"
DATA_DIR = environ.get('DATA_DIR', DEFAULT_DIR)

Are you pretty sure about some constant value you’re setting, but you may change it in the future? Optional ENV Variable is a slam dunk. You don’t even have to export the variable, you can define it on the same line as your command.

1

DATA_DIR=/var/data/ python myscript.py

Usernames, Passwords, sensitive data? You got it, ENV Vars.

1
2

USER = environ.get("USER")         # defaults to the username on the computer
PASS = environ.get("MY_APP_PASS")  # not saved into github anymore!!

You’ll get bored of setting these quickly, and miss the biggest strength of env vars if you don’t have your env vars set for you as you open your terminal. Look in your $HOME directory (ls -al $HOME) and you’ll find “rc” files. Maybe .zshrc or .bashrc. These configuration files are executed whenever you open a new terminal. You can put any code you’d like run before your sessions start in these files. export your ENV Vars here!

1
2

# ~/.zshrc
export SERVICE_HOST=whatever.com

Now you can always refer to $SERVICE_HOST in your terminals or programs! Save it once, use it always!

By separating the configuration of your app from your code you will be able to quickly reconfigure your code (am I pointing to my local machine, staging or production?) and more easily share config dependent code with others!