Today, I would not recommend anybody to go into graphics programming:
I started in 2001, when NVidias first Geforce 1 ("the Gigatexl shadercard") was first announced:
The field developed since then with so much speed and innovations, it blows my mind of. Compared to what we could do 25years ago, the tech today is just fu*ing impressive.
Though, with this impressiveness comes a big "but": The space is developing at a speed which is really really scary. Nvidia came up with AI-based effects to influence scene & assets on their own - back then, we wouldnt have even thought about that this will be possible some day in realtime.
I do not know if its possible at all to be a "decent pro" in this field now - let me use other words: "Where is todays Jon Carmack?" - he was famous for squeezing everything out of the hardware, using ideas very hidden in the community etc. - today, there is not any competitive moat for people like him (he actually lives on his legacy), and that is because the field is so vast and evolving so fast that there is no chance to become the next one
I really dislike people that got into a thing and then try to discourage others. “Don’t be like me! I wasted my entire life” which is bullshit from a jaded person that lost passion. Telling people to stay away from graphics programming is not how to entice tomorrow’s John Carmack.
So here’s another perspective. If all you have done is web apps and Kubernetes, for example, do get into graphics programming. The feedback cycle is exhilarating, and you get to appreciate how mind boggingly fast your average computer is. You’ll get to optimize things that are ultimately unimportant because you have never learned how quick things are at the low level. There are a ton of resources and the maths is not too bad. You might find that 3D modeling is a creative outlet you didn’t know you needed. Even if completely inapplicable to your day job, you’ll find new ways to appreciate the art of programming computers, and might just decide to never touch Kubernetes again and spend the next 5 years writing your own game engine in your spare time. There are a lot of crazy people like that, and the community of hobbyists that are not ground down by life and game dev as a career is larger than you’d think. The Graphics Programming discord is a welcoming place if you want to check it out.
That's not the argument being made here. The field is changing. I had a good career in graphics, my life wasn't wasted at all. That doesn't mean a college student would have the same experience starting today.
Well, of course not, unless you are claiming that a future career in graphics is a bad idea, and there is no way you could say that with any reasonable certainty, I do not get the argument at all.
The field is always changing. You could find people in the 80s saying ‘I had a good career in graphics, a college student would not have the same experience starting today’
For example, "there's no chance to become the next one" implies it's only worth it to do something if you can become the absolute best person in the field.
It's a big world. Most of us will not be the very best at what we do. There are millions of fun games that were not written by John Carmack.
> I really dislike people that got into a thing and then try to discourage others. “Don’t be like me! I wasted my entire life” which is bullshit from a jaded person that lost passion. Telling people to stay away from graphics programming is not how to entice tomorrow’s John Carmack.
Given that almost everyone who wants to be a "graphics programmer" is also somehow gaming industry adjacent, it is extremely fair to ward them off from the folly. I do the same for anyone wishing to do "VLSI hardware engineering." If you have the skill to do either of those, you almost CERTAINLY have the skill to do something else that is almost as interesting and not saddled by garbage employers.
The primary problem with being a "graphics programmer" beyond a tyro is that the biggest consumer of graphics programmers is the game industry which is a notoriously shitty and wretched industry. Every ... single ... employer. So, from the point of view of future potential, "graphics programmer" has very little upside over pretty much ANY other type of programmers.
Second, "learning graphics programming" is like "learning phone programming", you spend more time fighting godawful software infrastructure more than you do actual programming. AI actually kind of helps this, but it doesn't completely remove the fact that 80% of your knowledge has a half-life of 18-24 months.
Finally, saying "I want to learn graphics programming" is like saying "I want to learn engineering." What "graphics programmer" means is vastly underspecified. 3D game rendering and 3D/2D CAD rendering and 2D vector rendering are completely different skillsets. GPUs are great at the first and kinda okay at the second and kinda lousy at the third. Which kind of "graphics programmer" are you even going to be?
Graphics programming has this one very, very useful aspect, exponentially more valuable today: the matrix algebra pipelines, and then the requirement to 'think in matrix transforms' is a wonderful and visually engaging way to get your foundation for machine learning math.
I don't really see this with modern graphics programming, but I was highly amused that my 1980s-1990s graphics skills (in particular, coordinate transform math) were very useful when I started working in robotics in the 2010s-2020s (because forward and inverse kinematics are exactly the same thing as 2d/3d projections.)
The trick there is that they both have related physical analogs, and machine learning math doesn't really (in that while you can visualize them spatially, it doesn't seem to help solve any problems in that space.)
I mainly learned linear algebra via hands-on 3D graphics, and have a hard time thinking about a matrix as anything other than 4x4 and representing a linear transform...
How much do you even think about explicit matrix math when doing high-level ML?
I’ve not done high level ML, but I’ve done introductory ML and the truth is while the input space and the output space can have N and M dimensions, there’s not a lot of constraints involved. The matrix there are more randoms.
The whole ML field is basically about starting from random points and trying to find useful shapes and constraints. Basically like trying to get object likeness in clouds
there is absolutely no sense in which the SVD/PCA decomposition is just a rotation matrix. you should probably review your linear algebra textbook (hint: scaling is extremely important).
I'm a data scientist and not a graphics programmer, but my guess is that it's just abstracted away more. If you're using ML/DL libraries, you're mostly just calling APIs that handle the linear algebra and calculus for you. Unless you're actively contributing to those libraries, you likely don't ever need to "touch" any of the underlying operations. Up to a point, it's useful to understand how things work under the hood, but where that point is kinda depends on your job. For instance, I could write code to do 'naive' matrix multiplication, but I couldn't, like, contribute to BLAS.
... and I have been both situations for longer and have seen tons and tons of them (*)... So?
Not so hypotheticals -- Heck the inputs that you want labelled could be rotation matrices. The desired output could be a rotation matrix. Generating more convenient features could be via a rotation matrix. Dimensionality reduction could be through a reduction matrix. Sparsity could be encouraged by proper use of rotation matrices. Shows up if you want to build in group theoretic invariance in your predictive model.
How about people like Inigo Quilez? I'd say they're still quite high profile in today's landscape. And the main thing is I think there's just way more people in the field overall today too, not everyone can be famous! It's totally fine to not be as high profile as literally one of the most well known people in a field, it's fine to just do it because you enjoy it! The math and art of graphics (and games in general) programming is beautiful in and of itself.
I think the people that go into this field today (and for a while now) probably do it for the love of it, not the pay or widespread fame of doing something extraordinary in the field. Not that you can't have it all, but not being some legend I, well, I think that's a strange reason for someone already interested in game dev not to step into the field?
You can still read a bunch of papers and be first to market using exotic tech. The main issue right now is that games are so incredibly high budget and the bar is so high that you really need to stand out in many ways.
We see folks posting photo real, Gaussian splat FPS maps here every now and then but without also innovating on gameplay its just a tech demo. Those don't cut it these days.
Old 3D engine guy here. I highly encourage folks to make a 3D engine for fun and learning. Shipping a game with it would be a cherry on top. Come join us in r/graphicsprograming, r/gameenginedevs and the graphics programming discord.
It's also a great way to not become the next "graphics programming legend" --I think a fast-moving field with lots of new developments is actually an exciting place to be a pro.
JC was a bit of an anomaly but also his image is mostly coming from players and journalists. Developers struggled to use the later id software engines (partly why UE won that war).
You don’t need to be JC to earn a decent living as a graphics/game programmer.
Somewhat surprising there is no mention of basic design principles, or understanding the quirks of human perception. My brother was a production artist for some well-known computer games in the '90s-'00s, and continually complained about programmers and managers with zero visual sense, or curiosity about understanding the artists' side.
Graphics aren't my specialty, but as a musician, sound designer and producer, by far the most effective/influential audio DSP coders I'm aware of understand the basics of music, the physics/acoustics of sounds, and the gotchas at the interface between discrete digital processes and how we perceive and interpret stimuli.
I was a technical artist for a series of feature films during the early '00s. At a good studio they'll have art and design classes for the tech origin staff and scripting and bash classes for the art origin staff. I was both, and that was a ton of fun.
Exactly, Technical Artist is a distinct position that normally bridges the gap between pure programmers and artists and their needs. All TAs I've ever worked with had this incredible skill of knowing exactly what tech thing they need to achieve the outcome that the artists want.
Good Technical Artists are one of the most sought after professions in game dev. But it's also an annoyingly broad role that means different specific things at different places. The one common trait is being able to bridge the gap between art and code in a way that makes both parties happy.
This applies outside of creative industries too. I've seen my fair share of B2B/enterprise software where its clear the vendor has no clue how the industry they are selling to works, or how the users of that software think.
AI changed the calculus a bit (or at least, it has the potential to) but I think that was a huge part of the whole "learn to code" movement in the mid 2000s, to start treating software development as a "feature, not a product" of existing experts in their field so that the people most familiar with their domain are actually the ones making the software instead of having to translate the requirements down to a dev team.
Think more code monkeys for enterprise software consultancies, like Accenture, Tata, IBM Global Services, etc.
They needed warm bodies for their projects, as the usual source of manpower was grinding leetcode to work on bigtech at salaries that would make an accenture business type vomit in disgust.
(author of the article)
100% agree. As others have said, a good graphics programmer works with tech artists and artists. Frankly, graphics programming is largely a role of service to enable those people to do what they want to do, or help create what they envision.
People mentioned Inigo Quilez as an example of a graphics programmer who is also an artist. He is a power house and a unicorn.
I personally like playing music / programming audio more, which is a good ground for learning DSP things - useful when for instance, you want to push your rendering noise into the high frequencies, so a low pass filter is more effective at denoising.
I created and still maintain A-Frame (aframe.io). It’s been a gentle gateway to learn 3D graphics for a decade. Cool community if I can say so ha ha. Web is a great way to share stuff as you learn, collect feedback and get visibility. Many cases in the community of people that ended up doing 3d graphics professionally.
Start with just <a-box> and <a-sky>, add some animations, then add some community components if it's not enough. Still not enough then modify via ThreeJS, all the way to shaders. A-Frame is amazing so thanks for creating and maintaining it!
Feels like we try to turn anything we do into a career or job, especially with the odd ML angle. How about you "do graphics programming" instead of "being a graphics programmer"? Like start doing simple stuff until it clicks and you see it for being logistics to the GPU, then you can layer on top all the crazy concepts. Its like a small mountain you climb and suddenly everything clicks and you think like "oh my"... the possibilities and things to experiment with...
I would say being a long-time user of Photoshop and Blender helps a lot. It's not a main tool, but supplemental. Maybe AI will take over some of this though.
Why outsource my learning to Al? The whole point is the joy of the process. I could easily take a photo of a scene (since the inception of photography) instead of learning to paint it, but I would gain no skills through that. People still paint. I'm just tired, boss... I yearn for a past when we didn't have to end every conversation with a disclaimer about Al taking over.
trigonometry->Coordinate Geometry -> Linear Algebra applied to graphics
Once you have that intuition, the rest is all figuring out the stages of the graphics pipeline and the frameworks like opengl and their constituent data structures.
The most useful resources I've found for graphics are scratchapixel, UC Davis' graphics lectures, songho's articles, and Essential Math for Games and Interactive Applications. I highly recommend you read this last resource front to back. Seriously, its the best freaking math reference for graphics out there.
But knowing theory is not sufficient. You also need to get your hands dirty by writing code: learn how to build a software rasterizer (check out Tiny renderer) and a ray tracer (Ray Tracing in One Weekend series). Preferably in a language like C++. Then move onto APIs. I recommend you learn OpenGL, but if you're okay with being confused as all hell try Vulkan. Or WebGPU if you're a hipster (/s).
Finally, try to build some stuff. A simple engine. A non-trivial technique. A game. Whatever.
Unfortunately, you're unlikely to get hired working as a rendering engineer without having serious connections, or by having adjacent experience in the industry. Doubly so now that everyone is convinced junior engineers are unnecessary.
Thank you! I started recently as a greybeard engineer, and I found SDL3 GPU to be modern enough yet not too low level for a newbie compared to Vulkan. SDL in general is a fantastic framework. And if you use it from Odin, the dev experience is so smooth and enjoyable as everything you need to create graphical applications is builtin.
I will definitely check out Essential Math for Games and Interactive Applications, I feel I need some solid understanding of theory to see how it all fits together.
I started in 2001, when NVidias first Geforce 1 ("the Gigatexl shadercard") was first announced: The field developed since then with so much speed and innovations, it blows my mind of. Compared to what we could do 25years ago, the tech today is just fu*ing impressive.
Though, with this impressiveness comes a big "but": The space is developing at a speed which is really really scary. Nvidia came up with AI-based effects to influence scene & assets on their own - back then, we wouldnt have even thought about that this will be possible some day in realtime.
I do not know if its possible at all to be a "decent pro" in this field now - let me use other words: "Where is todays Jon Carmack?" - he was famous for squeezing everything out of the hardware, using ideas very hidden in the community etc. - today, there is not any competitive moat for people like him (he actually lives on his legacy), and that is because the field is so vast and evolving so fast that there is no chance to become the next one
So here’s another perspective. If all you have done is web apps and Kubernetes, for example, do get into graphics programming. The feedback cycle is exhilarating, and you get to appreciate how mind boggingly fast your average computer is. You’ll get to optimize things that are ultimately unimportant because you have never learned how quick things are at the low level. There are a ton of resources and the maths is not too bad. You might find that 3D modeling is a creative outlet you didn’t know you needed. Even if completely inapplicable to your day job, you’ll find new ways to appreciate the art of programming computers, and might just decide to never touch Kubernetes again and spend the next 5 years writing your own game engine in your spare time. There are a lot of crazy people like that, and the community of hobbyists that are not ground down by life and game dev as a career is larger than you’d think. The Graphics Programming discord is a welcoming place if you want to check it out.
Go for it!
That's not the argument being made here. The field is changing. I had a good career in graphics, my life wasn't wasted at all. That doesn't mean a college student would have the same experience starting today.
The field is always changing. You could find people in the 80s saying ‘I had a good career in graphics, a college student would not have the same experience starting today’
The rapid advances, in a trend replicating throughout society, push out the middle in favor of the top.
For example, "there's no chance to become the next one" implies it's only worth it to do something if you can become the absolute best person in the field.
It's a big world. Most of us will not be the very best at what we do. There are millions of fun games that were not written by John Carmack.
Given that almost everyone who wants to be a "graphics programmer" is also somehow gaming industry adjacent, it is extremely fair to ward them off from the folly. I do the same for anyone wishing to do "VLSI hardware engineering." If you have the skill to do either of those, you almost CERTAINLY have the skill to do something else that is almost as interesting and not saddled by garbage employers.
The primary problem with being a "graphics programmer" beyond a tyro is that the biggest consumer of graphics programmers is the game industry which is a notoriously shitty and wretched industry. Every ... single ... employer. So, from the point of view of future potential, "graphics programmer" has very little upside over pretty much ANY other type of programmers.
Second, "learning graphics programming" is like "learning phone programming", you spend more time fighting godawful software infrastructure more than you do actual programming. AI actually kind of helps this, but it doesn't completely remove the fact that 80% of your knowledge has a half-life of 18-24 months.
Finally, saying "I want to learn graphics programming" is like saying "I want to learn engineering." What "graphics programmer" means is vastly underspecified. 3D game rendering and 3D/2D CAD rendering and 2D vector rendering are completely different skillsets. GPUs are great at the first and kinda okay at the second and kinda lousy at the third. Which kind of "graphics programmer" are you even going to be?
The trick there is that they both have related physical analogs, and machine learning math doesn't really (in that while you can visualize them spatially, it doesn't seem to help solve any problems in that space.)
I've been in ML for ~5 years in multiple FAANGs and I have never seen a rotation matrix.
How much do you even think about explicit matrix math when doing high-level ML?
The whole ML field is basically about starting from random points and trying to find useful shapes and constraints. Basically like trying to get object likeness in clouds
Not so hypotheticals -- Heck the inputs that you want labelled could be rotation matrices. The desired output could be a rotation matrix. Generating more convenient features could be via a rotation matrix. Dimensionality reduction could be through a reduction matrix. Sparsity could be encouraged by proper use of rotation matrices. Shows up if you want to build in group theoretic invariance in your predictive model.
(*) If you consider Householders then even more
We see folks posting photo real, Gaussian splat FPS maps here every now and then but without also innovating on gameplay its just a tech demo. Those don't cut it these days.
You don’t need to be JC to earn a decent living as a graphics/game programmer.
Graphics aren't my specialty, but as a musician, sound designer and producer, by far the most effective/influential audio DSP coders I'm aware of understand the basics of music, the physics/acoustics of sounds, and the gotchas at the interface between discrete digital processes and how we perceive and interpret stimuli.
I think graphics programmers benefit from having an artistic mindset, but they usually work so low level that it isn’t necessary to be successful.
AI changed the calculus a bit (or at least, it has the potential to) but I think that was a huge part of the whole "learn to code" movement in the mid 2000s, to start treating software development as a "feature, not a product" of existing experts in their field so that the people most familiar with their domain are actually the ones making the software instead of having to translate the requirements down to a dev team.
They needed warm bodies for their projects, as the usual source of manpower was grinding leetcode to work on bigtech at salaries that would make an accenture business type vomit in disgust.
I also have some notebooks with SymPy code examples here: https://github.com/minireference/noBSLAnotebooks
Start with just <a-box> and <a-sky>, add some animations, then add some community components if it's not enough. Still not enough then modify via ThreeJS, all the way to shaders. A-Frame is amazing so thanks for creating and maintaining it!
PS: Oh, and you can even do AR and VR with it.
This guy has some good art: https://www.khanacademy.org/profile/kaid_1019042693170894950...
If you want to work with Windows, probably DirectX.
2. Make awesome shaders. Check this out: https://fragcoord.xyz
I would say being a long-time user of Photoshop and Blender helps a lot. It's not a main tool, but supplemental. Maybe AI will take over some of this though.
Hell, maybe that other stuff too, hahaha!
Once you have that intuition, the rest is all figuring out the stages of the graphics pipeline and the frameworks like opengl and their constituent data structures.
The most useful resources I've found for graphics are scratchapixel, UC Davis' graphics lectures, songho's articles, and Essential Math for Games and Interactive Applications. I highly recommend you read this last resource front to back. Seriously, its the best freaking math reference for graphics out there.
But knowing theory is not sufficient. You also need to get your hands dirty by writing code: learn how to build a software rasterizer (check out Tiny renderer) and a ray tracer (Ray Tracing in One Weekend series). Preferably in a language like C++. Then move onto APIs. I recommend you learn OpenGL, but if you're okay with being confused as all hell try Vulkan. Or WebGPU if you're a hipster (/s).
Finally, try to build some stuff. A simple engine. A non-trivial technique. A game. Whatever.
Unfortunately, you're unlikely to get hired working as a rendering engineer without having serious connections, or by having adjacent experience in the industry. Doubly so now that everyone is convinced junior engineers are unnecessary.
I will definitely check out Essential Math for Games and Interactive Applications, I feel I need some solid understanding of theory to see how it all fits together.