Nate Finch
@natefinch.bsky.social
Gopher at GitHub working on Copilot. Author of Mage. Dungeon Master and Magic player. Used to be NateTheFinch on the Bird Site that shall not be named.
Without unique error codes, all the client can do is display your error message and give up in the dumbest manner possible.
With error codes, the client can behave more intelligently - maybe show a dialog to request access to the directory, maybe show the list of trade-restricted countries, etc.
With error codes, the client can behave more intelligently - maybe show a dialog to request access to the directory, maybe show the list of trade-restricted countries, etc.
November 10, 2025 at 5:39 PM
Without unique error codes, all the client can do is display your error message and give up in the dumbest manner possible.
With error codes, the client can behave more intelligently - maybe show a dialog to request access to the directory, maybe show the list of trade-restricted countries, etc.
With error codes, the client can behave more intelligently - maybe show a dialog to request access to the directory, maybe show the list of trade-restricted countries, etc.
For example, maybe you return 403 forbidden - is that because the user made a request from a Trade Restricted country, or because they tried to read a directory they don't have permission for, or because they tried to delete a file they have read-only access to?
November 10, 2025 at 5:39 PM
For example, maybe you return 403 forbidden - is that because the user made a request from a Trade Restricted country, or because they tried to read a directory they don't have permission for, or because they tried to delete a file they have read-only access to?
Responding with json allows you to definite any number of unique error codes to fit your service's specific needs. This is exactly like returning well-known error types in Go. Clients can programmatically handle known errors, rather than just displaying an error message and giving up.
November 10, 2025 at 5:39 PM
Responding with json allows you to definite any number of unique error codes to fit your service's specific needs. This is exactly like returning well-known error types in Go. Clients can programmatically handle known errors, rather than just displaying an error message and giving up.
Define a standard json body schema that *all* endpoints implement, so that clients can build that into their response handling logic. This makes their error handling logic a lot more robust, and means the API authors have a lot more flexibility in how you can communicate errors to clients.
November 10, 2025 at 5:39 PM
Define a standard json body schema that *all* endpoints implement, so that clients can build that into their response handling logic. This makes their error handling logic a lot more robust, and means the API authors have a lot more flexibility in how you can communicate errors to clients.
If a client gets a 400, they need to know *how* the request was bad, so they can convey it to the user. They should get a body something like this:
{
"error" : {
"code" : "invalid_filename_characters",
"field" : "filename",
"message" : "filenames cannot contain emoji"
}
}
{
"error" : {
"code" : "invalid_filename_characters",
"field" : "filename",
"message" : "filenames cannot contain emoji"
}
}
November 10, 2025 at 5:39 PM
If a client gets a 400, they need to know *how* the request was bad, so they can convey it to the user. They should get a body something like this:
{
"error" : {
"code" : "invalid_filename_characters",
"field" : "filename",
"message" : "filenames cannot contain emoji"
}
}
{
"error" : {
"code" : "invalid_filename_characters",
"field" : "filename",
"message" : "filenames cannot contain emoji"
}
}
Yep. 🤷♂️. I'm not on the infra team. They're aware of the problem. There are some solutions like this being bandied about.
October 31, 2025 at 2:46 PM
Yep. 🤷♂️. I'm not on the infra team. They're aware of the problem. There are some solutions like this being bandied about.
For loaded API servers, slower processing means more requests in flight at once which means more RAM allocated until each request is finished.
And that's how we figured out we needed more CPU resources to prevent nodes from running out of RAM.
And that's how we figured out we needed more CPU resources to prevent nodes from running out of RAM.
October 31, 2025 at 1:44 PM
For loaded API servers, slower processing means more requests in flight at once which means more RAM allocated until each request is finished.
And that's how we figured out we needed more CPU resources to prevent nodes from running out of RAM.
And that's how we figured out we needed more CPU resources to prevent nodes from running out of RAM.
Now let's say slow machines take 6 seconds to process each request, but they're still getting 50 requests per second. So the slow machines have 300 requests in process at any one time. At 10 megs per request, the slow machines have to allocate 3000 megs at a time.
1.5x as much as the fast machines.
1.5x as much as the fast machines.
October 31, 2025 at 1:44 PM
Now let's say slow machines take 6 seconds to process each request, but they're still getting 50 requests per second. So the slow machines have 300 requests in process at any one time. At 10 megs per request, the slow machines have to allocate 3000 megs at a time.
1.5x as much as the fast machines.
1.5x as much as the fast machines.
That means the fast machines have 200 requests in flight at any one time. Let's say each request uses on average 10 megs of RAM. That means the fast servers are using 2000 megs at a time.
October 31, 2025 at 1:44 PM
That means the fast machines have 200 requests in flight at any one time. Let's say each request uses on average 10 megs of RAM. That means the fast servers are using 2000 megs at a time.
It's all about throughput.
The root of the problem is that traffic is distributed evenly among the machines. So let's say each machine gets 50 new requests every second. Let's say the average request on a fast machine takes 4 seconds to process (remember, this is AI, it ain't fast).
The root of the problem is that traffic is distributed evenly among the machines. So let's say each machine gets 50 new requests every second. Let's say the average request on a fast machine takes 4 seconds to process (remember, this is AI, it ain't fast).
October 31, 2025 at 1:44 PM
It's all about throughput.
The root of the problem is that traffic is distributed evenly among the machines. So let's say each machine gets 50 new requests every second. Let's say the average request on a fast machine takes 4 seconds to process (remember, this is AI, it ain't fast).
The root of the problem is that traffic is distributed evenly among the machines. So let's say each machine gets 50 new requests every second. Let's say the average request on a fast machine takes 4 seconds to process (remember, this is AI, it ain't fast).
So it turns out, for 🤷♂️ reasons, GH's kubernetes nodes get distributed across machines of different capabilities. The machines all have access to the same amount of RAM, but with faster or slower CPUs.
We figured out that the slower machines were the ones consistently running out of RAM. But why?
We figured out that the slower machines were the ones consistently running out of RAM. But why?
October 31, 2025 at 1:44 PM
So it turns out, for 🤷♂️ reasons, GH's kubernetes nodes get distributed across machines of different capabilities. The machines all have access to the same amount of RAM, but with faster or slower CPUs.
We figured out that the slower machines were the ones consistently running out of RAM. But why?
We figured out that the slower machines were the ones consistently running out of RAM. But why?
We noticed some nodes of our deployment of GitHub's Copilot API (which runs on top of Kubernetes) were spiking memory 1.5x or more than the average and eventually getting OOM (out of memory) killed.
But why only some nodes? Were they caused by bad/malicious requests that ballooned RAM usage? Nope.
But why only some nodes? Were they caused by bad/malicious requests that ballooned RAM usage? Nope.
October 31, 2025 at 1:44 PM
We noticed some nodes of our deployment of GitHub's Copilot API (which runs on top of Kubernetes) were spiking memory 1.5x or more than the average and eventually getting OOM (out of memory) killed.
But why only some nodes? Were they caused by bad/malicious requests that ballooned RAM usage? Nope.
But why only some nodes? Were they caused by bad/malicious requests that ballooned RAM usage? Nope.
Every time you do this, you disrupt *every other developer* on the project.
Sure it's "just add xyz to your .env" but inevitably there's a weird error message and you spend an hour investigating, thinking someone introduced a bug.... multiplied by half the dev team, that's a lot of lost effort.
Sure it's "just add xyz to your .env" but inevitably there's a weird error message and you spend an hour investigating, thinking someone introduced a bug.... multiplied by half the dev team, that's a lot of lost effort.
October 24, 2025 at 5:04 PM
Every time you do this, you disrupt *every other developer* on the project.
Sure it's "just add xyz to your .env" but inevitably there's a weird error message and you spend an hour investigating, thinking someone introduced a bug.... multiplied by half the dev team, that's a lot of lost effort.
Sure it's "just add xyz to your .env" but inevitably there's a weird error message and you spend an hour investigating, thinking someone introduced a bug.... multiplied by half the dev team, that's a lot of lost effort.
It's *so* aggravating to git pull or git merge main and all of a sudden get bizarre errors when I try to build/run the code that was *just working*. It feels like someone managed to check in a bug.
"Have you tried recreating the codespace / docker container / updated your .env file?"
(╯°□°)╯︵ ┻━┻
"Have you tried recreating the codespace / docker container / updated your .env file?"
(╯°□°)╯︵ ┻━┻
October 24, 2025 at 5:04 PM
It's *so* aggravating to git pull or git merge main and all of a sudden get bizarre errors when I try to build/run the code that was *just working*. It feels like someone managed to check in a bug.
"Have you tried recreating the codespace / docker container / updated your .env file?"
(╯°□°)╯︵ ┻━┻
"Have you tried recreating the codespace / docker container / updated your .env file?"
(╯°□°)╯︵ ┻━┻
Seems a shame they're not a 4 mana 4/4. Feel like they could have been sans-white and been a bit more on-theme. I wish WotC wouldn't jump to 5 colors so easily.
October 10, 2025 at 4:45 PM
Seems a shame they're not a 4 mana 4/4. Feel like they could have been sans-white and been a bit more on-theme. I wish WotC wouldn't jump to 5 colors so easily.
Nope. And I don't particularly love UB and wish everything was magic IP. But don't split the format.
Also, basically everyone at my LGS, who are probably less online than me and anyone in this thread, don't care at all about UB one way or another. To them, they're all just cards.
Also, basically everyone at my LGS, who are probably less online than me and anyone in this thread, don't care at all about UB one way or another. To them, they're all just cards.
October 7, 2025 at 12:02 PM
Nope. And I don't particularly love UB and wish everything was magic IP. But don't split the format.
Also, basically everyone at my LGS, who are probably less online than me and anyone in this thread, don't care at all about UB one way or another. To them, they're all just cards.
Also, basically everyone at my LGS, who are probably less online than me and anyone in this thread, don't care at all about UB one way or another. To them, they're all just cards.
That's exactly how I feel.
I loved Clark's vulnerability. That interview with Lois was SUCH a good scene. Clark's exasperation and frustration felt *so relatable*. I think that's the real magic – this was the most relatable Superman I've ever seen.
"PEOPLE WERE GOING TO DIE, LOIS!"
*chef's kiss*
I loved Clark's vulnerability. That interview with Lois was SUCH a good scene. Clark's exasperation and frustration felt *so relatable*. I think that's the real magic – this was the most relatable Superman I've ever seen.
"PEOPLE WERE GOING TO DIE, LOIS!"
*chef's kiss*
September 30, 2025 at 1:35 PM
That's exactly how I feel.
I loved Clark's vulnerability. That interview with Lois was SUCH a good scene. Clark's exasperation and frustration felt *so relatable*. I think that's the real magic – this was the most relatable Superman I've ever seen.
"PEOPLE WERE GOING TO DIE, LOIS!"
*chef's kiss*
I loved Clark's vulnerability. That interview with Lois was SUCH a good scene. Clark's exasperation and frustration felt *so relatable*. I think that's the real magic – this was the most relatable Superman I've ever seen.
"PEOPLE WERE GOING TO DIE, LOIS!"
*chef's kiss*
REST forces everything into CRUD... except some things aren't crud so what you get is 80% crud and 20% "I dunno just use POST". Just skip the middle man.
Http methods are extraneous for APIs. You have the url as the method name. POST /getFoo and POST /updateFoo are perfectly clear.
Http methods are extraneous for APIs. You have the url as the method name. POST /getFoo and POST /updateFoo are perfectly clear.
September 5, 2025 at 1:44 AM
REST forces everything into CRUD... except some things aren't crud so what you get is 80% crud and 20% "I dunno just use POST". Just skip the middle man.
Http methods are extraneous for APIs. You have the url as the method name. POST /getFoo and POST /updateFoo are perfectly clear.
Http methods are extraneous for APIs. You have the url as the method name. POST /getFoo and POST /updateFoo are perfectly clear.
HTTP verbs very often don't make sense. It's the old "what verb represents 'transfer $100 from this account to that account'"?
Real APIs aren't just CRUD.
It's just like a library's API. There are a bunch of methods (endpoints) and you need to read docs to figure out how to use them.
Real APIs aren't just CRUD.
It's just like a library's API. There are a bunch of methods (endpoints) and you need to read docs to figure out how to use them.
September 4, 2025 at 5:22 PM
HTTP verbs very often don't make sense. It's the old "what verb represents 'transfer $100 from this account to that account'"?
Real APIs aren't just CRUD.
It's just like a library's API. There are a bunch of methods (endpoints) and you need to read docs to figure out how to use them.
Real APIs aren't just CRUD.
It's just like a library's API. There are a bunch of methods (endpoints) and you need to read docs to figure out how to use them.
Oh and while we're at it, return structured (json) errors with unique error code separate from HTTP status codes from day one. Don't rely on status codes to be informative enough. They never are.
September 4, 2025 at 4:24 PM
Oh and while we're at it, return structured (json) errors with unique error code separate from HTTP status codes from day one. Don't rely on status codes to be informative enough. They never are.
you really REALLY do not need to follow REST. Everything is POST with a JSON body is perfectly fine for an API that never gets directly hit by a browser. Just treat it like any other JSON API, except it happens to use HTTP as the transport.
September 4, 2025 at 4:24 PM
you really REALLY do not need to follow REST. Everything is POST with a JSON body is perfectly fine for an API that never gets directly hit by a browser. Just treat it like any other JSON API, except it happens to use HTTP as the transport.