MOTOSHARE 🚗🏍️

Turn Idle Vehicles into Income

Owners Earn. Riders Move. Motoshare Connects.

Start with Motoshare

Eight Months of ContainerOps: A Composite Trading-Analytics Team

Eight Months of ContainerOps: A Composite Trading-Analytics Team
Trading Technology

You will not find this team anywhere, because it was invented for the purpose. Think of it as a composite sketch: five people who run a small analytics service for retail traders, a charting backend and a nightly screener, on two rented servers. Their market opinions are beside the point here, and nothing below is trading or investment advice. What matters is how they learned to look after the software containers that carry their service, a craft the Ops Atlas calls ContainerOps. The numbers they meet are borrowed from the Atlas’s own reference material; the people and their calendar are made up.

Month zero: the setup everybody recognises

The service started as a weekend project. A developer wrote a build recipe, ran it on a laptop, and tagged the result “latest”. Servers pulled that tag. The containers ran as the root user because nothing complained. There were no memory limits, no log settings and no list of what ran where. By the Atlas’s five-level yardstick, which begins with laptop builds, floating tags and root users, they sat at the bottom rung, and nothing felt wrong because nothing had gone wrong yet.

The Atlas page, the ContainerOps entry in the Ops Atlas, is blunt that this stage is normal. It is where most projects begin.

Month two: the first scare

The screener job started missing its overnight window. One night the container was killed, and the monitor showed exit status 137. The lead developer assumed a bug. A teammate recalled that 137 is 128 plus 9, the forced-kill signal, and asked a better question: killed by whom? The answer, from the host’s own logs, was the memory controller. The job had grown, and with no ceiling set, it had competed with the charting service for memory until the kernel chose a victim.

Their first fix was small: set a memory ceiling for each container and a cap on process counts. The Atlas recommends doing exactly that for every container, while treating CPU limits more carefully. They left CPU alone for the moment.

Month three: the day the supply moved

Then came the more instructive event. A base image the team used was among a family of free images whose publisher announced that the free catalogue would move away from its public home. Updates would cease in the old location. Anyone not paying would only get a floating latest tag. The team had a Friday build planned and discovered, mid-deploy, that the image reference they relied on would go stale.

They survived because one member had, almost by accident, kept a local copy in their own registry. The retrospective listed three lessons. Know which of your base images come from sources you do not control. Mirror what production needs. And reference images by their digest, so any substitution would be visible instead of silent.

Month four: pinning without freezing

Pinning by digest raised an objection from the most cautious member. “If we pin everything, we will never get fixes.” The worry is reasonable, and the Atlas answers it head-on: it is only true when no bot is involved. They installed an update bot that proposes new base digests as pull requests. Each proposal ran through the build, a short test, and a review. In the first month, the bot opened nine proposals. They merged seven and rejected two after a test failure, which is how it is supposed to work.

From then on, “we deployed whatever was latest” became “we deployed digest such-and-such, which was reviewed on this date”. For a service whose users care about accuracy and uptime during market hours, that provenance trail mattered.

Month five: the quiet security work

They next looked at how their containers ran. The Atlas lists a handful of habits and they adopted them in order: a non-root numeric user, all capabilities dropped, a read-only root filesystem, and ports bound to the loopback address behind a single reverse proxy. The last change caught something. A database port had been published with just a number, which meant it listened on all interfaces, and the host firewall rules they thought protected it did not apply to published ports the way they assumed. Nothing had been abused, as far as they could tell, but the discovery changed how seriously they treated the checklist.

Month six: measuring what they had

The team picked three measures from the Atlas and put them on a whiteboard: the share of containers referenced by digest, the age of their oldest base image, and the delay from an upstream release of a fix to the moment it runs in production. The first stood at a little over eighty percent, which is the Atlas’s goal for its second month of work. The second was embarrassing. For the third they set a target of one week for serious issues, borrowing the Atlas’s illustration, and then missed it twice before reworking their pipeline.

That is the point of numbers. They do not flatter; they point.

Month seven: a slow chart that was not slow

A trader-facing chart endpoint began answering unevenly just before market open. CPU graphs looked calm, with usage averaging comfortably under the cap they had finally applied. A teammate who had read the Atlas’s throttling example knew what to look for: the counter of throttled scheduling periods against total periods. It showed forty-odd percent throttled. The average hid the stalls, exactly as in the Atlas’s worked case, where a two-CPU cap and a modest average still left a large share of periods throttled.

There was a second cause hiding behind the first. The service’s runtime sized its thread pool from the number of processors it could see, and inside a container that is the host’s count, not the allowance. Setting the pool size explicitly and lifting the cap slightly brought the ratio under five percent, the figure the Atlas gives for latency-sensitive work.

Month eight: stopping politely and logging sensibly

Deployments had always felt sluggish. Each release left the old container hanging on for ten seconds before vanishing. The entrypoint used shell form, so the stop request never reached the program and the engine eventually forced it. Moving to exec form and adding a tiny init process cut release times noticeably and removed a class of half-written nightly files. At the same time they discovered that log files on one server had grown to fill a third of the disk, because the default driver does not rotate them. A size cap and a file count fixed it in an afternoon.

What the story teaches

  1. Normal beginnings are untidy, and that is fine as long as you move on.
  2. Kill signals have causes you can read; do not guess.
  3. Own a copy of what production cannot do without.
  4. Pin, but let a bot keep you moving.
  5. Tighten privileges in small, reviewed steps.
  6. Measure a few things honestly and review them on a schedule.
  7. Read the averages sceptically: a calm graph can still conceal stalls.
  8. Teach programs to stop politely and cap their own logs.

What the team would tell a newcomer

Asked for advice by a friend who runs a similar one-server service, the lead developer gave three sentences. Write down what you run before you change anything. Fix the loudest problem first, which is usually memory or exposure, and resist the temptation to polish the quiet ones. Pick a date for the next review before the current one ends. None of it is clever, and the team likes that about it; a calm checklist cost them fewer sleepless nights than the heroic debugging sessions of month two ever did.

Connecting it to trading discipline

The parallels with a trader’s own habits are hard to miss. A written plan with defined exits resembles a pinned, reviewed deployment, and a step-by-step approach to keeping control of your trades through a disciplined routine is the same mindset applied to a different craft. The same goes for the calm preparation described in a piece about starting the trading day with a steady morning routine. In both, boring consistency beats heroics.

The composite team still has untidy corners. They have not yet run a full rebuild drill, and one old tool runs on a runtime line near the end of its support. But the calendar now lists those items, each with an owner. That, more than any single setting, is what changed.

0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x