Benchmark: CadQuery vs. OpenSCAD for agentic CAD work

(modelrift.com)

13 points | by jetter 1 hour ago

5 comments

  • fallat 3 minutes ago
    such a garbage post for various reasons, but the ending is the kicker

    after making SEVERAL observations that cadquery is pretty much better overall, they say "ehhh we're going to stay with openscad".

  • dofm 46 minutes ago
    None of the "findings" here really needed an AI experiment to do it:

    - "CadQuery fails loudly and early... OpenSCAD fails silently and late"

    This should be obvious, from the documentation, from the way geometry construction works. OpenSCAD has no sense of failure where one shape means another won't work; you're just drawing the equivalent of 3D pixels in space. It will always potentially-meaninglessly succeed if the syntax is OK.

    - Renders caught nothing that mattered

    Objects with hollows are not going to show their major problems this way.

    - CadQuery can be interrogated, OpenSCAD cannot

    Isn't this explicit from the documentation of both? One works by iteratively building on top of a previous result that can be stored in a variable; the other doesn't.

    - OpenSCAD renders have no concept of a part edge

    Again — this should be explicit from the documentation, which describes no methods for operating on edges (and largely from the fact that it is declarative).

    - Speed favours OpenSCAD, and it barely matters

    Yes, being faster is no good if things are wrong.

    - "What decides it is verifiability rather than expressiveness, and CadQuery leads there by a wider margin than the syntax difference suggests."

    Yes, because the difference is semantic. Which you can get from the documentation. Building iteratively on the basis of existing geometry is inherently more verifiable, because stuff that can't work won't work.

    Honestly do people not try learning CAD before they try to build an AI tool to generate CAD models? This feels like yet another situation where people who have not done the foundational conceptual work seem to think that they can avoid it and just work around it with AI.

  • ariwilson 56 minutes ago
    not interesting as the writeup as very AI sloppish :(. i'm sad about that as I would really like to have an intuitive sense for how good the current frontier models are at coming up with interesting 3d printable parts.
    • ____tom____ 47 minutes ago
      I've been just getting started on this with Astra. Astra is one-shotting good 3D models from photographs for me. I was unable to get Sol to do this, even after multiple rounds.

      It's not a small change, it's from "not useful" to "pretty good"

      • elwebmaster 39 minutes ago
        With any tool?
      • dofm 45 minutes ago
        > Astra is one-shotting good 3D models from photographs for me.

        Right, but, so what? There is very little utility here. Parts that need to be useful need to be designed to spec.

        • ____tom____ 34 minutes ago
          Maybe not for you, but I'm building miniatures for custom board games. Works great.
          • dofm 29 minutes ago
            Right but there is no meaningful specification failure mode in creative CAD, is there? I mean, yes, image to CAD for a little miniature, I can buy that it is OK. Fun, even, especially when your use is sort of transitory.

            But this article is talking about functional parts. A functional part exists because of its spec, usually fails because of some combination of its material, manufacture, and spec. And the spec is chosen with that in mind.

            Even working from a text description of a part is error prone because language is imprecise and interpretable. I just don't understand why AI people want to race past actual proper sensible solutions to a problem just to automate generating from text.

            • elwebmaster 20 minutes ago
              People don't have time (or even mental capability / even more time) to learn CAD but they have a task at hand which they want done. If AI can generate a part which is "good enough" for the user at a cost that the user considers reasonable (I.e. zero) then AI gets used. If a part fails thats rarely a concern because these parts are just 3d printed and likely cost single digit cents in filament. User can always print a new one.
              • dofm 13 minutes ago
                Right, but the thing that I think sort of militates against this scenario is this:

                If a non-CAD person imagines a part that they need, the chances are very high indeed that either it isn't what they need, can't be made robustly or sensibly, or if it is, it already exists in some meaningful way that can be adapted, because one of tens of millions of other people already wanted it.

                This scenario — needing something very specific that can be described only in words and nobody else has ever needed before — is almost imaginary. Most stuff just isn't like that.

                Now… if someone were to design an AI CAD package that could remix objects that exist and combine them, perhaps with a GUI tool that allowed a user to identify attachment points, maybe that would be interesting. But a lot of that has been solved in non-AI GUIs already.

                Text-to-CAD just isn't as good an idea as it seems. You are not going to be able to magic up some novel invention from a text description without a significant amount of domain knowledge of the words needed to describe a part, and that domain knowledge usually comes from learning CAD!

                (Bit like getting a coding agent to write really good code)

                And simple parts, sorry, just don't need much CAD skill. Get a CAD package on a free plan, learn enough to make your thing. It'll probably have standard tools for adding threads, even.

                Even image-to-CAD for a functional part requires a significant amount of understanding of what to photograph and from which angles, etc.

        • Mashimo 38 minutes ago
          Once you have the model, it's relativ easy to change the demensions / constrains later on, no?
          • dofm 36 minutes ago
            Not if the model has not been implemented from reasonable principles.
            • Mashimo 15 minutes ago
              What does that mean in practice?
    • rjsw 51 minutes ago
      There was a blog post from a couple of days ago that had been initially flagged, it seemed to me to have a good overview of AI for CAD, will try to find the link.
    • dofm 23 minutes ago
      [dead]
  • tamimio 32 minutes ago
    Should try different models and harnesses, CC and opus5 aren’t the best combo. Also, I expect soon to have models that verify the mesh and even render it not just generating the code. So we got this for mechanical engineering, cupperhead and similar for electronics, usual agents for software, what is next?
  • brcmthrowaway 38 minutes ago
    The OP seems to be shilling this tool nonstop. Modelrift just seems to be an LLM wrapper.. the original models can handle OpenSCAD just fine.