Now that I’ve walked through the history of OpenGL and seen what I can do at what points in history, I can attack one of the long-standing problems I had with my old pixmap library. It’s not easy to see the issue with the logo image I’ve been using, so I’ll be replacing it with some simple 1-bit checkerboard patterns to make it clearer. If I take a 256×256 checkerboard and scale it with the system we developed over the last two updates, some weird extra patterns emerge. We get a sort of secondary checkerboard falling out of what’s supposed to be an even grid:
Zooming in on a “corner” of that secondary checkerboard makes it clear what is happening:
Since we’re not rendering an even multiple of the 256×256 image here, the renderer is picking the nearest texel for each pixel. Each “checkerboard” pattern expands from a 2×2 block to something that isn’t quite a 3×3 block, and as the fractional parts of each pair of pixels adds up, we ultimately switch, in each dimension, from two black pixels to one white pixel over to two white per one black.
The usual solution to this is to switch to bilinear filtering of the image, to blend each pixel into its neighbors so that these intermediary pixels are grey. This isn’t hard: in our code we just alter the assignments of GL_TEXTURE_MAG_FILTER and GL_TEXTURE_MIN_FILTER from GL_NEAREST to GL_LINEAR . That, however, becomes a problem if we drop the underlying image from 256×256 all the way down to, say, 16×16:
Bilinear filtering begins blending at the center of each texel, which means that when each texel is 30 or so pixels across, they collapse into a blurry mess. What we want is to do this blending but only on pixels that encompass two of our texels, and blend only there. We should be zooming in to something like this:
and then that will look nice and sharp when we have a full display:
The most common name for this kind of magnification system seems to be sharp bilinear filtering. I’ve wanted one of these for awhile, but I’ve never gotten around to actually implementing any. In my previous reading I found some solutions that are almost immediately usable.
Scaling Pixel Art Without Destroying It, a 2017 blog post by Cole Cecil, is where I started. The shader built here is for the Unity game engine, and its format is markedly different than what I’ve been using. Somebody told me about this article long ago and I have completely forgotten who it was. Thank you in absentia!
Manual Texture Filtering for Pixelated Games in WebGL at the Algorithmic Pensieve blog inspired the Cole Cecil post, and it’s working with three.js and WebGL, a tech stack closer to my own.
Superior texture filter for dynamic pixel art upscaling is a followup article to the previous, correcting some blurring issues that could occur when the previous technique was used alone.
For this article, I will re-derive the mathematics behind the shader and then port the logic into my engines that ran under OpenGL 2.1 and OpenGL 3.2.
The Basic Theory
Here’s the fragment shader we were using before, essentially a reskin of the old fixed-function pipeline:
The core idea of our new scaler is while we’re handed texLoc by the vertex shader, and while we’re handing it directly over to the texture2D() function as texture coordinates, we aren’t actually required to do this. When we sample the texture, we can sample it anywhere we want. The overall plan, then, goes like this:
Set bilinear interpolation mode. This means that if we magnify a texture, each pixel in a texel will vary between “50/50 blend of this texel with its neighbor” at the edges and “the actual stated color of the texel” in the center.
When shading each pixel, figure out where in our source texel we actually are.
If we’re sufficiently in the middle of the texel, sample from the dead center of the texel instead of our actual fragment location.
Otherwise, we’re in a border area of the texel. Vary between the 50/50 blend and the main texel color based on how deep we are into this border rather than how deep we are into the texel itself. It turns out that we can also do that by sampling from a suitable coordinate within the texel’s space instead of our actual fragment location.
We define “the border area” to basically just include any pixel that multiple texels would contribute to. If we choose it so that it is only a single pixel wide, we’d be perfectly blending the contributions of all participating texels. In practice, it seems to look better if we make the border area just a little bit wider than that—if we make it too sharp we end up with the same sorts of “shimmering” effects nearest-neighbor gives us.
The articles I’ve linked include some very nice graphs of the functions they’re working with and why they work, but I did notice that they don’t quite derive where the final formula come from; they just annotate the charts. I’m going to actually work through this here.
Working Through the Math
We have two variables here:
The fractional texel coordinate, which is 0 at the left or top edge, 1 at the right or bottom edge, and 0.5 in the center.
The blend factor, a 50/50 mix with our left/top neighbor at 0, a 50/50 mix with our right/bottom neighbor at 1, and no mixing at all at 0.5.
In each dimension, we’ll know our fractional texel coordinate (it’ll be the input to the fragment shader) and we need to produce the blend factor (which will inform what we’ll pass to sample2D() ). We’ll also know how wide the border is, as a fraction of the total texel size. I’ll follow Algorithmic Pensieve here and call this width α. If α is 0, there is no border at all (and we’re basically doing nearest-neighbor), and if α is 0.5, then we are all border (and we’re basically doing bilinear filtering).
Our formula for the blend factor will end up being a piecewise linear function that passes through four points as the fractional texel coordinate moves from 0 to 1:
(0, 0). We’re a 50/50 blend at the edge.
(α, 0.5). Once we leave the border region, we are at our texel’s unblended color.
(1-α, 0.5). We hold that unblended color until we enter the border region on the other side.
(1, 1). We’re a 50/50 blend at the other edge too.
This produces three line segments, which, by the usual “rise over run” definition, have slopes of either (0.5 / α) in the borders or 0 in the center. From there we could derive the rest of the linear equations, and then write a shader that checks the fractional texel coordinate against our thresholds. However, before we do that, Algorithmic Pensieve makes a useful optimization: fancy control flow in shaders is generally expensive, but more controlled forms of conditionality are often much cheaper. In this case, we can rely on the “clamping” operation that forces values to within a range, and turn our graph into the sum of two clamped linear functions:
This is basically the same line twice, just shifted. The first line runs through the origin, so given that we know the slope, the leftmost graph is a clamped version of the equation y=(0.5/α)x. The second line runs through the point (1, 0.5), so we can translate the line right 1 unit and up half a unit to yield the equation y=(0.5/α)(x-1)+0.5. Despite the shape being very different, it is nevertheless clamped to the same range as the first.
Computing α
We still have a challenge here, though: we really want the blend area to be measured in pixels, and α is measured in texels. The “right value” for α will be different depending on exactly how much magnification we are doing.
Happily, this is a very easy problem. If we have a particular border width β in pixels, we simply multiply that value by the number of texels-per-pixel at our current magnification to to get α. Cole Cecil’s code does this by hand, working out the scaling factor as, effectively, a uniform, but it turns out that GLSL will give us this for free.
Fragment shaders in OpenGL have access to some special functions named dFdx() and dFdy() which are a little fragile but effectively treat their argument as if it were a function instead of a value and determine the difference of that function between its value in this fragment and the value with the previous fragment in the relevant dimension. If we want both at once, there is also a function fwidth() with extremely confusing documentation in the GLSL spec but which seems to make these two statements equivalent:
(EDIT, 11 Oct 2025: I’d mischaracterized the translation of fwidth on vec2 values, and I think I have it straightened out here. The actual dFdx() -based sample code in the linked articles up top will struggle with rotated sprites but the fwidth -based solutions should be able to handle it. Thanks to Vorn for talking me through the issues here.)
Algorithmic Pensieve relies on these, but writing in 2014, also noted that if you wanted to use these in WebGL (itself a variant of OpenGL ES 2), you’d need to ensure that certain extensions were available in your environment. But I am writing in 2025, a time where we have WebGL 2, which is instead based on OpenGL ES 3, and which guarantees the existence of all three of these functions.
More to the point, right here and now we are using OpenGL proper and not OpenGL ES or WebGL and the main line of OpenGL has had these functions for as long as it has had fragment shaders in the core specification. We are in no way compromising compatability when we rely on these.
Updating the Shaders
We have a little bit of work left to do when translating all this into shader code.
First, all our work above has been in terms of fractional texel coordinates. We don’t get those as input; in fact, we don’t get texel coordinates at all. We get values that range from 0 to 1 irrespective of the size of the texture. We need to convert from and to those ranges to the exact texel we’re interested in and our fractional coordinate within it. Functions like fract() and floor() can manage splitting out the integral and fractional parts of texel coordinates, but we’ll need to pass in the size of our actual texture to do the full work.
Second, we’ve only really been dealing with a single dimension. We need to handle both horizontal and vertical blending for this system. Happily, GPU programming makes this really easy. Each dimension is independent, and almost any mathematical operation can be done to vectors as easily as it can to single numbers. These operations either replicate a single number and operate on it with every element of the vector, or take two vectors and do the operation on each dimension independently. Our shaders will mostly look like we’re doing the computation once but on vector values instead of scalar ones.
I made two versions of the final shader, one for OpenGL 3.2 which should work in any of the modern APIs, and one for OpenGL 2.1 for systems like my old Raspberry Pi that refuses to create OpenGL 3.x contexts. There isn’t much difference between them.
OpenGL 3.2
The vertex shader is almost identical to our simple shader from last week that did nothing more than draw the image. All we have to add is the translation of our texture coordinates into texel coordinates. This requires passing in the texture dimensions as a uniform and then multiplying our texture coordinates by them on the way out.
The fragment shader is where we do the actual work we described. It also needs the texture dimensions to translate back to the 0-1 range that OpenGL demands, and it also accepts the length of the blend region β in pixels as a scalar uniform. Here’s the code:
The shader body has four lines:
The first line computes the slope of the lines in our graph above: (0.5/α). The fwidth() function carries out the necessary translation from pixels to texels.
function carries out the necessary translation from pixels to texels. The second line extracts the fractional part of the texel coordinates we received as input from the vertex shader.
The third line computes scaled , the sum of the two clamped linear functions that was the focus of all our work.
, the sum of the two clamped linear functions that was the focus of all our work. Finally, we get the output color by treating that result as our new fractional texel coordinate, adding it to the integral part of our texture coordinate then dividing it by the texture dimensions to renormalize to the 0-1 range. Once that’s been done we may feed it to texture2D() as if this is ordinary bilinear filtering.
That’s all it takes! This was the shader I used to produce the images at the top of the article.
OpenGL 2.1
When I upgraded my way through OpenGL’s history, I made a point of altering as little as possible in each version to showcase the continuity between the old fixed-function APIs and the modern shader-based ones. This meant that the OpenGL 2.1 code was presented as a very slight transformation of the pre-existing OpenGL 1.5 code. There’s no reason to do that here; for this question, we’d like to keep the rendering code reasonably similar between the legacy and modern APIs.
From our final 2.1 shader last time, that means relying on only two of the values in gl_Vertex and using the scale uniform instead of the ModelView transform. For this week’s project, the only real differences between the 2.1 and 3.2 are the #version directive, the attribute declarations, and whether we use gl_Vertex or a custom attribute:
The fragment shader changes even less. The only differences are the #version directive and the texLoc parameter being labeled varying .
Updating the Framework Code
We don’t need to alter much about our simple code from last time here; there’s only three major changes, and only one is actually a surprise.
We have some new uniforms. We need to collect their indices in renderInit() and give them appropriate values in render() .
and give them appropriate values in . We need to change the magnification and minification filters to GL_LINEAR so that our shader’s interpolation works the way we intend it to.
so that our shader’s interpolation works the way we intend it to. Finally, the surprise change: we need to change the GL_TEXTURE_WRAP parameters from GL_CLAMP to GL_CLAMP_TO_EDGE . This is a new feature that was added in OpenGL 1.2.
The original GL_CLAMP option turns out to be basically useless. Here’s what happens if we leave it as GL_CLAMP and then try to scale the display:
That tiny border is because when scrolling past the edge of the texture it doesn’t actually clamp to the edge of the texture; it instead locks itself to a single per-texture color value you theoretically configure as a texture parameter. GL_CLAMP_TO_EDGE makes the actual edges of the texture spread out forever which is the actually useful behavior for things like this. The only reason we’d never been burned by this before was because we were using GL_NEAREST and thus never made it out of range in the first place.
Beyond Pixel Maps
We’ve been talking about all of this in the context of scaling a single textured rectangle, but what we’ve actually implemented goes well beyond this. Thanks to fwidth() automatically adapting to anything the vertex shader hands us, and thanks to the texture dimensions being updatable as easily as the actual texture bound to the sampler, any textured graphical primitive may be scaled up sensibly in this way. Rotated or even 3D-projected textures will still respond in relatively sensible ways when shaded in this manner—though they might not actually look like low-res pixel art afterwards! We are playing in a liminal space here, between low-resolution source graphics and a variety of approaches to displaying them on high-resolution displays.
We seem to have already gotten to the point where pixel-art scaling and sharp bilinear filtering are an old enough technique that people will simply name it and assume that everyone who matters knows what this is. That’s always a danger with these sorts of effects—half my demoscene notes over the years have been trying to reconstruct old techniques that we’ve kind of forgotten how to teach to novices—but hopefully this reference can add to the historical ones.