Jason Nochlin
@nochlin.com
Working on a follow-up to my shell script UX post. What's the most impressive shell script UX feature you've encountered in the wild?
March 5, 2025 at 5:44 PM
Working on a follow-up to my shell script UX post. What's the most impressive shell script UX feature you've encountered in the wild?
Thanks for the share @gunnarmorling.dev ☺️
February 16, 2025 at 4:32 AM
Thanks for the share @gunnarmorling.dev ☺️
I'm not sure if there's a cleaner way, pretty new to the tool 🙂
December 24, 2024 at 9:43 PM
I'm not sure if there's a cleaner way, pretty new to the tool 🙂
Here's an example of a simple pipeline configured with Dagster
![from dagster import job, op, Definitions, SkipReason, schedule
from datetime import datetime
import requests
@op(retry_policy={
"max_retries": 3,
"delay": 60 # 1 minute between retries
})
def check_api_state():
response = requests.get("https://api.example.com/status")
data = response.json()
# Return None if conditions aren't met, which will skip downstream ops
if data["status"] != "READY" or data["queue_size"] > 100:
return None
return data
@op
def process_data(context, api_state):
# This won't run if api_state is None
if not api_state:
context.log.info("Conditions not met, skipping processing")
return
# Your processing logic here
context.log.info(f"Processing data with state: {api_state}")
@job
def conditional_job():
state = check_api_state()
process_data(state)
@schedule(
job=conditional_job,
cron_schedule="0 * * * *" # Run hourly
)
def hourly_schedule():
return {}
defs = Definitions(
jobs=[conditional_job],
schedules=[hourly_schedule]
)](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:zxykpb36liefxnp4bmax4ro6/bafkreifdeqkkjrmlic66yodruhneeuwj3hzpyzwizostb7ff65yj47k2ji@jpeg)
December 24, 2024 at 9:25 PM
Here's an example of a simple pipeline configured with Dagster
I recently evaluated orchestration tools and landed on Dagster b/c it seemed the simplest (can run on a SQLite DB) and is configured with code. Planning to use the open-source version and host it myself, but haven't implemented yet.
December 17, 2024 at 5:12 PM
I recently evaluated orchestration tools and landed on Dagster b/c it seemed the simplest (can run on a SQLite DB) and is configured with code. Planning to use the open-source version and host it myself, but haven't implemented yet.