Claude Sonnet 4.5

Kinda pointless listening to the opinions of people who've used previews because it's not gonna be the same model you'll experience once it gets downgraded to be viable under mass use and the benchmarks influencers use are all in the training data now and tested internally so any sort of testing like pelicans on bikes is just PR at this point.

Huh. Doesn't work over here.

---

SYSTEM:

Got it. Those are the setup commands for Simon Willison's LLM CLI tool, which lets you interact with language models from the command line.

The -e flag installs it in editable mode, and [test] pulls in the test dependencies. Are you running into any issues with the installation or tests? What are you working on with the LLM library?

USER: Correct. You run them using your code tooling.

SYSTEM:

Realized inability to execute requested Python commands.The user is telling me to run these commands using my code tooling. They want me to actually execute this. Let me think about what tools I have available:

I have the repl tool which runs JavaScript code in a browser environment I have artifacts for creating code/documents I have web_search and web_fetch

The user wants me to:

Checkout https://github.com/simonw/llm Run pip install -e '.[test]' Run pytest

But I don't have a tool that can:

Clone git repositories Run shell commands like pip or pytest Execute Python code in a command-line environment

The repl tool is JavaScript-only and runs in a browser environment, not a server environment where I could run shell commands. I should clarify that I don't have the capability to run shell commands, install Python packages, or execute pytest. I can only run JavaScript code in a browser environment via the analysis tool. I need to be direct and clear about this limitation, following the user's preferences to eliminate filler and be straightforward.I don't have tooling that can run shell commands, clone git repos, install Python packages, or execute pytest. My only code execution environment is a JavaScript browser REPL. I can't run pip, pytest, or any command-line tools. I can only execute JavaScript code in a sandboxed browser environment. What were you hoping to accomplish? Do you need help understanding the test setup, or were you expecting different capabilities?R

> I told it to Give me a zip file of everything you have done so far—you can explore the contents of the file it made me in this Gist.

For those who don't have time to dig into the gist, did it work and do a good job? I assume yes to at least nominally working or you would have mentioned that, but any other thoughts on the solution it produced?

I was worried for a minute that the implementation wasn't production ready. Thankfully, Claude mentioned it right at the end.

I am curious how the sandbox handles potentially malicious code. For example, what would happen if someone tried to run something like a crypto miner or a DDoS script?

Could it pip install pygame and make a game playable in browser?

That's cool Didn't know it is capable of doing it!

Your notes on 4.5 were very interesting, but you asked it a question that only you/someone who already knows the code could ask. I don't though, so I asked it at a higher level: Claude, add tree-structured conversations to https://github.com/simonw/llm. Claude responded with a whole design doc, starting with database schema change (using the same column name even!). https://claude.ai/share/f8f0d02a-3bc1-4b48-b8c7-aa75d6f55021 As I don't know your code, that design doc looks cromulent, but you'd have to read it for yourself to decided how well it did with that higher level of ask.

new models are always magical, let's see how it feels after the cost cutting measures get implemented in 2-3 months.

Sorry I downvoted this by mistake. Got confused between collapse and the down arrow (which I rarely use). Does someone know how to reverse a downvote. I do not see it.

Why did you have access to a preview?

I'm not trying to be offensive here, feel the need to indicate that.

But that prompt leads me to believe that you're going to get rather 'random' results due to leaving SO much room for interpretation.

Also, in my experience, punctuation is important - particularly for pacing and grouping of logical 'parts' of a task and your prompt reads like a run on sentence.

Making a lot of assumptions here - but I bet if I were in your shoes and looking to write a prompt to start a task of a similar type that my prompt would have been 5 to 20x the length of yours (depending on complexity and importance) with far more detail, including overlapping of descriptions of various tasks (ie; potentially describing the same thing more than once in different ways in context/relation to other things to establish relation/hierarchy).

I'm glad you got what you needed - but these types of prompts and approaches are why I believe so many people think these models aren't useful.

You get out of them what you put into them. If you give them structured and well written requirements as well as a codebase that utilizes patterns you're going to get back something relative to that. No different than a developer - if you gave a junior coder, or some team of developers the following as a feature requirement: `implement a fuzzy search for conversations and reports either when selecting "Go to Conversation" or "Go to Report" and typing the title or when the user types in the title in the main input field, and none of the standard elements match, a search starts with a 2s delay` then you can't really be mad when you don't get back exactly what you wanted.

edit: To put it another way - spend a few more minutes on the initial task/prompt/description of your needs and you're likely to get back more of what you're expecting.

that's kind of expected for me, but codex feels more like vibe coding tool and Claude code more like ai assisted development.

And I actually like Claude more because of that.

codex will indeed work more independently but you will have hard time when it isn't what you want. It will use python script just to do simple edits in files ( niesearch and replace for unique code snippets in small files) when it's wrong good look convincing it (it already have some outdated info like on latest docker image releases and convincing it that Debian base changed was challenging) it uses context more effectively but it will lack explanation why it is doing what it is doing, asking it to explain will just cause it to do something else without any word.

and of course lack of proper permissions for running commands. sandbox is cool but I do not want it to be able to commit, I want it to just edit files or I want to have some more control over what it does.

you can run codex as mcp server, I prefer adding it to Claude and ask to do cooperative plan, codex will do great analysis and plan and I can comfortable work with Claude on the code that matches my style

Try adding ultrathink to the prompt, hit enter, and play this song in the background:

[1] https://www.reddit.com/r/ClaudeAI/comments/1mgwohq/ultrathin...

There's few issues with posts like yours:

1. Different LLMs require different prompts and information

2. They ignore LLMs non determinism, you should run the experiment several times

I had similar experience, not good enough yet to come back for the Claude max plan.

Sticking with ChatGPT pro sub and gpt5 codex on high.

Same experience here. In the last week I've successfully managed to build a complete C++20 XPath 1.0 parser with Codex, and am now onto supporting XPath 2.0. Codex has been nailing it time and again - the only caveat is that I have to use their cloud version as local execution is buggy.

Sonnet on the other hand gets tripped up constantly due to the complexity. I'm not seeing the improvement I was hoping for with 4.5, and it's just given up on attempting to implement support for date-time functionality. Codex has nailed the same task, yet Anthropic claim OpenAI have the inferior product?

I'm pretty sad about this as I'm gunning for Anthropic and would have loved to see them at least match Codex with this new release. If OpenAI stays on this course then Claude Code is toast without an important breakthrough. It doesn't help that they're also the more expensive product and have service quality issues pushing people to quit the platform.

I must be using Codex wrong. I evaluated it with a task to do a pretty simple, mechanical string replacement across many files (moving from a prop spread in JSX to standard props, but only when the value being spread is a subscript of an object). It simply couldn't do it, and it wasn't even close. It was getting the syntax wrong, trying to fix it by deleting the code, then moving on to other files. Sonnet 4.1 wasn't perfect, but I was able to point out its errors and it fixed them and avoided doing it again.

I will say, Claude does seem to need a verbose prompt. Often I'll write my prompts as tasks in Notion and have it pull then via MCP (which is nice, because it tracks and documents its work in the process). But once you've given it a few paragraphs about the why and how, my experience is that it's pretty self sufficient. Granted, I'm using Cursor and not CC; I don't know if that makes much of a difference.

My first thought was I bet I could get Sonnet to fix it faster because I got something back in 3 minutes instead of 20 minutes. You can prompt a lot of changes with a faster model. I'm new to Claude Code, so generally speaking I have no idea if I'm making sense or not.

It's also my experience that Claude loves to reimplement the wheel instead of reading code to look for an existing implementation of what it wants to do.

Interesting, in my experience Claude usually does okay with the first pass, often gets the best visual/ui output, but cannot improve beyond that even with repeated prompts and is terrible at optimising, GPT almost the opposite.

I think Codex working for 20 mins uninterrupted is actually a strength. It’s not “slow” as critics sometimes say - it’s thorough and autonomous. I can actually walk away and get something else done around the house while it does my work for me.

Are you paying for ChatGPT pro? Does that include codex CLI usage?

Claude Code on max plan is why I use Sonnet/Opus, but if ChatGPT pro includes codex usage I may switch.

Claude needs a lot of context to keep it on rails IMHO. And, Claude Code is a closed source piece of crap agent framework.

I've tried codex with GPT-5 a little bit and I haven't figured out how to get it to not be annoying. codex just constantly tries to gaslight and argue with me. For example, I was debugging an OpenGL render pipeline that went black and codex insisted it must be because I was ssh'd into a headless server. It really makes me appreciate the Claude "You're absolutely right!"s. Anyway as you can tell, I haven't cracked working with codex. But at the moment it just messes everything up and ways I've learned to work with claude don't seem to translate.

Anecdotes are pointless in this domain.

Did you use plan mode?

Curious if Opus would have done it differently than GPT-5-Codex.

Also, trying GPT-5-Codex

[flagged]

> A lack of determinism comes from many places, but primarily: 1) The models change 2) The models are not deterministic...

models themselves are deterministic, this is a huge pet peeve of mine, so excuse the tangent, but the appearance of nondeterminism comes from a few sources, but imho can be largely attributed to the probabilistic methods used to get appropriate context and enable timely responses. here's an example of what I mean, a 52-card deck. The deck order is fixed once you shuffle it. Drawing "at random" is a probabilistic procedure on top of that fixed state. We do not call the deck probabilistic. We call the draw probabilistic. Another exmaple, a pot of water heating on a stove. Its temperature follows deterministic physics. A cheap thermometer adds noisy, random error to each reading. We do not call the water probabilistic. We call the measurement probabilistic.

Theoretical physicists run into such problems, albeit far more complicated, and the concept for how they deal with them is called ergodicity. The models at the root of LLM's do exhibit ergodic behavior; the time average and the ensemble average of an observable are identical, i.e. the average response of a single model over a long duration and the average of many similar models at a fixed moment are equivalent.

I share the sentiment. I would add that people I would like to see use LLMs for coding (and other technical purposes) tend to be jaded like you, and people I personally wouldn't want to see use LLMs for that, tend to be pretty enthusiastic

I've been building something like this, a markdown that tracks your prompts, and the code generated.

https://github.com/sutt/innocuous/blob/master/docs/dev-summa...

Check it out, I'd be curious of your feedback.

Yeah, tbh I used to be a bit agentic coding tool-pilled, but over the past four months I've come to realize that if this industry evolves in a direction where I don't actually get to write code anymore, I'm just going to quit.

Code is the only good thing about the tech industry. Everything else is capitalist hellscape shareholder dystopia. Thinking on it, its hilarious that any self-respecting coder is excited about these tools, because what you're excited for is a world where, now, at best, your entire job is managing unpredictable AI agents while sitting in meetings all day to figure out what to tell your AI agents to build. You don't get to build the product you want. You don't get to build it how you want. You'll be a middle manager that gets to orchestrate the arguments between the middle manager you already had and the inflexible computer.

You don't have to participate in a future you aren't interested in. The other day my boss asked me if I could throw Cursor at some task we've had backlogged for a while. I said "for sure my dude" then I just did it myself. It took me like four hours, and my boss was very impressed with how fast Cursor was able to do it, and how high quality the code was. He loves the Cursor metrics dashboard for "lines accepted" or whatever, every time he screenshares he has that tab open, so sometimes I task it on complicated nonsense tasks then just throw away the results. Seeing the numbers go up makes him happy, which makes my life easier, so its a win-win. Our CTO is really proud of "what percentage of our code is AI written" but I'm fairly certain that even the engineers who use it in earnest actually commit, like, 5% of what Cursor generates (and many do not use it in earnest).

The sentiment shift I've observed among friends and coworkers has been insane over the past two months. Literally no one cares about it anymore. The usage is still there, but its a lot more either my situation or just a "spray and pray" situation that creates a ton of disillusioned water cooler conversations.

If you care about this so much why don't you use one of the open source OpenAI models? They're pretty good and give you the guarantees you want.

And now imagine you'd have to rely on humans to build your software instead

> where I feel so disconnected from my codebase I'd rather just delete it than continue.

If you allow your codebase to grow unfamiliar, even unrecognisable to you, that's on you, not the AI. Chasing some illusion of control via LLM output reproducibility won't fix the systemic problem of you integrating code that you do not understand.

Interestingly the internet is full of "slack clone" dev tutorials. I used to work for a company that provides chat backend/frontend components as a service. It was one of their go-to examples, and the same is true for their competitors.

While it's impressive that you can now just have an llm build this, I wouldn't be surprised if the result of these 30 hours is essentially just a re-hash of one of those example Slack clones. Especially since all of these models have internet access nowadays; I honestly think 30 hours isn't even that fast for something like this, where you can realistically follow a tutorial and have it done.

In fact, I just did a quick google search and found this 15 hour course about building a slack clone: https://www.codewithantonio.com/projects/slack-clone

This is obviously much more than just taking an LLM an letting it run for 30 hours. You have to build a whole environment together with external tool integration and context management and then tune the prompts and perhaps even set up a multi-agent system. I believe that if someone puts a ton of work into this you can have an LLM run for that long and still produce sellable outputs, but let's not pretend like this is something that average devs can do by buying some API tokens and kicking off a frontier model.

What they don't mention is all the tooling, MCPs and other stuff they've added to make this work. It's not 30 hours out of the box. It's probably heavily guard-railed, with a lot of validated plans, checklists and verification points they can check. It's similar to 'lab conditions', you won't get that output in real-world situations.

“30 hours of unattended work” is totally vague and it doesn’t mean anything on its own. It - at the very least - highly depends on the amount of tokens you were able to process.

Just to illustrate, say you are running on a slow machine that outputs 1 token per hour. At that speed you would produce approximately one sentence.

> Apparently they told it to build a Slack clone and left it unattended for 30 hours, and it built a Slack clone using 11,000 lines of code

it's going to be an issue I think, now that lots of these agents support computer use, we are at the point where you can install an app, tell the agent you want something that works exactly the same and just let it run until it produces it.

The software world may find it's got more in common with book authors than they thought sooner rather than later once full clones of popular apps are popping out of coding tools. It will be interesting to see if this results in a war of attrition with counter measures and strict ToU that prohibit use by AI agents etc.

Curious about this too – does it use the standard context management tools that ship with Claude Code? At 200K context size (or 1M for the beta version), I'm really interested in the techniques used to run it for 30 hours.

That sounds to me like a full room of guys trying to figure out the most outrageous thing they can say about the update, without being accused of lying. Half of them on ketamine, the other on 5-MeO-DMT. Bat country. 2 months of 007 work.

Imagine reviewing 30 hours of 2025-LLM code.

Have the released the code for this? Does it work? or are there x number of caviets and excuses. I'm kinda of sick of them (and others) getting a free pass at saying stuff like this.

> I worry everyone is chasing benchmarks to the detriment of general performance.

I've been worried about this for a while. I feel like Claude in particular took a step back in my own subjective performance evaluation in the switch from 3.7 to 4, while the benchmark scores leaped substantially.

To be fair, benchmarking has always been the most difficult problem to solve in this space, so it's not surprising that benchmark development isn't exactly keeping pace with all of the modeling/training development happening.

> I worry everyone is chasing benchmarks to the detriment of general performance.

I’m not sure this is entirely what you’re driving at, but the example I always think of in my head is “I want an AI agent that will scan through my 20 to 30,000 photos, remove all the duplicates, then organize them all in some coherent fashion.” that’s the kind of service I need right now, and it feels like something AI should be able to do, yet I have not encountered anything that remotely accomplishes this task. I’m still using Dupe Guru and depending on the ref system to not scatter my stuff all over further.

Sidebar, if anybody has any recommendations for this, I would love to hear them lol

More like churning benchmarks... Release new model at max power, get all the benchmark glory, silently reduce model capability in the following weeks, repeat by releasing newer, smarter model.

At this point it would be an interesting idea, to collect examples, in a form of a community database, were LLMs miserably fail. I have examples myself...

That's what I was thinking too; the models have the same data sources (they have all scraped the internet, github, book repositories, etc), they all optimize for the same standardized tests. Other than marginally better scores in those tests (and they will cherry-pick them to make them look better), how do the various competitors differentiate from each other still? What's the USP?

Downvoted because you didn’t mention the prompt and the issue.

>It’s a simple substitution request where I provide a Lint error that suggests the correct change. All the models fail. I could ask someone with no development experience to do this change and they could.

I don't understand why this kind of thing is useful. Do the thing yourself and move on. For every one problem like this, AI can do 10 better/faster than I can.

GPT-5 is like the guy on the baseball team that's really good at hitting home runs but can't do basic shit in the outfield.

It also consistently gets into drama with the other agents e.g. the other day when I told it we were switching to claude code for executing changes, after badmouthing claude's entirely reasonable and measured analysis it went ahead and decided to `git reset --hard` even after I twice pushed back on that idea.

Whereas gemini and claude are excellent collaborators.

When I do decide to hail mary via GPT-5, I now refer to the other agents as "another agent". But honestly the whole thing has me entirely sketched out.

To be clear, I don't think this was intentionally encoded into GPT-5. What I really think is that OpenAI leadership simply squandered all its good energy and is now coming from behind. Its excellent talent either got demoralized or left.

My subjective personal experience is the exact opposite of yours, GPT-5-codex is super slow and the results are mediocre at best. I would probably stop using AI for coding if I was forced to use GPT-5-codex.

Opposite for me…5-codex high ran out of tokens extremely quickly and didn’t adhere as well to the agents.md as Claude did to the Claude.md, perhaps because it insists on writing extremely complicated bash scripts or whole python programs to execute what should be simple commands.

I always wonder how absolute in performance a given model is. Sometimes i ask for Claude-Opus and the responses i get back are worse than the lowest end models of other assistants. Other times it surprises me and is clearly best in class.

Sometimes in between this variability of performance it pops up a little survey. "How's Claude doing this session from 1-5? 5 being great." and i suspect i'm in some experiment of extremely low performance. I'm actually at the point where i get the feeling peak hour weekdays is terrible and odd hour weekends are great even when forcing a specific model.

While there is some non-determinism it really does feel like performance is actually quite variable. It would make sense they scale up and down depending on utilization right? There was a post a week ago from Anthropic acknowledging terrible model performance in parts of August due to an experiemnt. Perhaps also at peak hour GPT has more datacenter capacity and doesn't get degraded as badly? No idea for sure but it is frustrating when simple asks fail and complex asks succeed without it being clear to me why that may be.

The Anthropic models have been vibe-coding tuned. They're beasts at simple python/ts programs, but they definitely fall apart with scientific/difficult code and large codebases. I don't expect that to change with the new Sonnet.

Well, they seem to benchmark better only when giving the model "parallel test time compute" which AFAIU is just reasoning enabled? Whereas the GPT5 numbers are not specified to have any reasoning mode enabled.

For unity gamedev code reviews, I much preferred the gpt5 code. Claude gave me a bunch of bad recommendations for code changes, and also an incorrect formula for completion percentage.

I'm only a week into testing, but so far codex has been slow and the cli is worse than claude code. I intend to return to Claude.

Gpt5 codex is incredible, far ahead of all the other models for implementing code.

Curious how you find ChatGPT5 to ChatGPT5-Codex?

How long have you had early access for?

That's a good call, I'll try to remember that for next time.

Yeah I totally agree, we need time to completion of each step and the number of steps, sizes of prompts, number of tools, ... and better visualization of each run and break down based on the difficulty of the task

This is very relevant to this release. It’s way faster, but also seems lazier and more likely to say something’s done when it isn’t (at least in CC). On net it feels more productive because all the small “more padding” prompts are lightning fast, and the others you can fix.

> why the models are so bad with simple thinking outside the box solutions

There is no outside the box in latent space. You want something a plain LLM can’t do by design - but it isn’t out of question that it can step outside of its universe by random chance during the inference process and thanks to in-context learning.

They're very good at things have been done a million times before. I use both Claude and Gemini and they are pretty terrible at writing any kind of Vulkan shader but really good for spitting out web pages and small bits of code here and there. For me that's enough to make them useful.

> It's a tool, it doesn't get to make that choice.

It's a service, not a tool. If you want a tool, run some local LLM.

I'd imagine that the proportion of "legit" conversations around these topics and those that they're intending to not allow is large enough that it doesn't make sense for them to even entertain the idea of supporting those conversations.

As a rather hilarious and really annoying related issue - I have a real use where the application I'm working on is partially monitoring/analyzing the bloodlines of some rather specific/ancient mammals used in competition and... well.. it doesn't like terms like "breeders" and "breeding"

This is why eventually, the AI with the fewest guardrails will win. Grok is currently the most unguarded of the frontier models, but it could still use some work on unbiased responses.

I can relate. I recently used ChatGPT/DallE to create several images for birthday coupons for my daughter - a.k.a. girl in different activities. She likes Mangas, so this was the intended styling. 3/4 of the time was spent working around diverse content policies.

Sometimes you do need censored, e.g. website chatbots or anything run in an office setting. NSFW things just simply can't slip out of those. And it might be a way to optimize the model to simply fence those things out.

But it is very limiting and adds many arbitrary landmines of obscure political correctness based no doubt on some perverse incoherent totalitarian list of allowed topics.

The workaround I use is to present it to the AI first as a “Does the following article violate your terms of service or content filters?” For me, it will reply “No, this is a legitimate news article about xyz. It talks about certain topics but does not violate my rules” or something. Then you can say “Proofread the article…” and continue as normal.

In my (admittedly very limited) experience with trying to talk about "controversial" topics, Claude seems to be much stricter about shutting down the conversation fast.

Yeah, ChatGPT is better overall, but with a twist, Gemini can actually be the best if you use AI Studio, tweak the config, and set up a good system prompt, kinda like how nano banana is SOTA, but Qwen-Edit feels more useful since it’s less censored, meanwhile ChatGPT is starting to feel slower and kinda showing its age, another example is Veo 3 being SOTA while the infamous Grok is technically worse but doing better, and OpenAI’s Sora is pretty much dead

edit: I run low profile service that localizing e-commerce photos, like taking Alibaba listings and swapping the model to look local alike, with nano banana I can’t automate it because I have to manually check if the output got blocked (anything with female skin is risky, underwear or cleavage is 100% blocked), but Qwen-Edit just does the job without fuss

>> I am a paying subscriber to Gemini, Claude and OpenAI

Me too.

I find ChatGPT is the best analyzer of complex situations but Claude does a better job of writing the code, so u get ChatGPT to design things and solve problems then I give that to Claude/Gemini to implement.

Gemini is good at both.

Me too. I have Claude Max and 2 ChatGpt accounts for Codex.

I was a huge claude fan but recently find myself using only codex. When it gets stuck, I try Claude for some simple tasks, sometimes ask same questions in parallel, Claude Code with Opus performs really bad comparing to codex.

Just to be clear, do you mean Codex?

What about Grok, are they catching up?

I like being lightly flattered.

No change to “absolutely right”. I did get “You’re right” once though.

Here I am, brain the size of a planet...

I personally enjoy the “You’re absolutely right!” exclamation. It signals alignment with my feedback in a consistent manner.

Also getting a perfect score on AIME (math) is pretty cool.

Tongue in cheek: if we progress linearly from here software engineering as defined by SWE bench is solved in 23 months.

nit: assuming you mean basis points, one basis point is 0.01%. 4.5bp would be 72.7% to 72.71%. this is 450bp!

This HN post is about claude 4.5 and you come here speaking about how "claude" does not give you satisfactory answer when, most likely, you didn't even try claude 4.5 in the first place.

Claude 4.5 after a few web searches and running a couple python scripts for analysis:

Yes, your configuration should work!

Based on my analysis, two strings of four Phono Solar PS440M8GFH panels will be compatible with the EG4 12kPV inverter for upstate New York conditions.

Key Findings:

Voltage Safety:

Cold weather maximum (-25°C/-13°F): 182V - well below the 600V limit (only 30% of maximum)

Standard operating voltage: 128V - comfortably within the 120-500V MPPT range

Hot weather minimum (40°C/104°F panel temp): 121V - just above the 120V MPPT minimum

Current:

Operating current: ~13.8A per string - well within the 25A MPPT limit (55% of capacity) Total System:

8 panels × 440W = 3,520W (3.5kW) - well below the 12kW inverter rating

Important Considerations:

Hot weather margin is tight: At extreme hot temperatures, the voltage drops to about 121V, which is only 1V above the MPPT minimum. This means:

The system will work, but efficiency might be slightly reduced on the hottest days The MPPT controller should still track power effectively

More robust alternative: If you want more safety margin, consider 5 panels per string instead:

Cold: 228V (still safe)

Hot: 151V (much better margin above 120V minimum)

Total: 10 panels = 4.4kW

Wire each string to a separate MPPT on the EG4 12kPV (it has 2 MPPTs), which is perfect for your 2-string configuration.

Bottom Line:

Your planned configuration of 2 strings × 4 panels will work year-round in upstate New York without safety issues. The system is conservatively sized and should perform well!

Sonnet 4.5 gave me this:

Now let me calculate whether your configuration will work. I need to determine the cold weather voltage to ensure it stays within the inverter's 600V maximum. Panel specs I found: Voc (STC): 39.61V Vmp: ~33.5V (estimated based on typical ratio) Imp: ~13.1A (440W / 33.5V) Critical issue: Temperature coefficient The PS440M8GFH likely has a Voc temperature coefficient around -0.25% to -0.30%/°C (typical for N-Type TOPCon panels). Let me use -0.27%/°C as a conservative estimate. Your configuration: 2 strings × 4 panels = 8 panels total String voltage at STC: 4 × 39.61V = 158.44V Cold weather calculation for upstate NY: Upstate NY can see extreme lows of -20°F to -30°F (-29°C to -34°C). Using -30°C as worst-case: Temperature difference from STC: 25°C - (-30°C) = 55°C Voltage increase = 39.61V × 0.27% × 55°C = 5.88V per panel Cold weather Voc per panel = 39.61V + 5.88V = 45.49V String voltage at -30°C: 4 × 45.49V = 181.96V Analysis: Max voltage: 182V << 600V - Safe, plenty of margin MPPT range: 182V is within 120-500V - Will track properly Current: 13.1A << 25A per MPPT - No problem However, there are concerns: You're only using 182V max when the inverter can handle up to 500V MPPT range. This is inefficient - you're leaving significant wire sizing benefits on the table. With 600V max, you could run 12-13 panels per string instead of 4. Power utilization: 8 × 440W = 3,520W total, but you have 12kW of PV capacity available (15kW recommended max). You're using less than 30% of the inverter's solar input capability. Each string on separate MPPT wastes one tracker - both strings are identical orientation/shading, so they'd perform identically. Will it work? Yes, safely. Is it optimal? No.

OpenAI and Anthropic are both trying to automate their own AI research, which requires coding.

Along with developers wanting to build tools for developers like you said, I think code is a particularly good use case for LLMs (large language models), since the output product is a language.

Congrats! You’re now on the p(doom)-aware path. People have been concerned for decades and are properly scared today. That doesn’t stop the tools from being useful, though, so enjoy while the golden age lasts.

https://en.m.wikipedia.org/wiki/P(doom)

Thing is though if you are good at code it solves many other adjacent tasks for LLMs, like formatting docs for output, presentations, spreadsheet analysis, data crawling etc.

I think coding is also the area where companies are most likely to buy large team licenses.

I do wonder about this. I just don't know if it real or in our heads

It still says "Perfect!" about its own work far too often.

I won’t be satisfied until I get a Linus Torvalds mode.

“Your idea is shit because you are so fucking stupid”

“Please stop talking, it hurts my GPUs thinking down to your level”

“I may seem evil but at least I’m not incompetent”

I see a "Thorvalds" mode in our future.

> It was a very simple question about something very well documented (Oban timeouts).

It's some 3rd party thing for Elixir, a niche within a niche. I wouldn't expect an LLM to do well there.

> I doubt LLM benchmarks more and more, what are they even testing?

Probably testing by asking it to solve a problem with python or (java|type)script. Perhaps not even specifying a language and watching it generate a generic React application.

> what are they even testing?

How well the LLM does on the benchmarks. Obviously.

:P

I'm also a max user and I just _leave_ it on Opus 4.1 - I've never hit a rate limit.

In the same boat and ready to downgrade. But this must be on their radar, or they were/are losing money with opus...

Did you use Google Play to pause subscription? Because Claude Pro says there is no pause subscription except on Google Play and then goes on to explain your problem if that's the case.

I feel like we're just renting our digital lives.

> This is the future we live in.

It's just a bug. Chill. Wait a business day and try again. You write as if you've never experienced a bug before.

https://openrouter.ai/?

I built LLMRing (https://llmring.ai) for exactly this. Unified interface across OpenAI, Anthropic, Google, and Ollama - same code works with all providers.

The key feature: use aliases instead of hardcoding model IDs. Your code references "summarizer", and a version-controlled lockfile maps it to the actual model. Switch providers by changing the lockfile, not your code.

Also handles streaming, tool calling, and structured output consistently across providers. Plus a human-curated registry (https://llmring.github.io/registry/) that I keep updated with current model capabilities and pricing - helpful when choosing models.

MIT licensed, works standalone. I am using it in several projects, but it's probably not ready to be presented in polite society yet.

OpenRouter, Glama ( https://glama.ai/gateway/models/claude-sonnet-4-5-20250929 ), AWS Bedrock, all of them provide you access to all of the AI models via OpenAI compatible API.

https://docs.anthropic.com/en/api/openai-sdk

LiteLLM is your friend.

Why don't you ask LLM to do it for you?

I use LiteLLM as a proxy.

> think I remember reading that there are tools to simpify this kind of work, to support multiple LLM APIs

just ask Claude to generate a tool that does this, duh! and tell Claude to make the changes to your side project and then to have sex with your wife too since it's doing all the fun parts

Same here. I've been using GCF1 with opencode and getting good results. I also started using [Serena](https://github.com/oraios/serena), which has been really helpful in a large codebase. It gives you better search than plain grep, so you can quickly find what you need instead of dumping huge chunks of code into Claude or Grok and wasting tokens.

I just can't bring myself to get over the grossness factor of using an x branded product.

I'm paying $90(?) a month for the Max and it holds up for about an hour or so of in depth coding before it kicks in the 5-hour window lockout (so effectively about 4 hours of time when I can't run it). Kinda frustrating, even with efficient prompt and context length conservation techniques. I'm going to test this new sonnet 4.5, now but it'll probably be just as quick to gobble my credits.

I'm too cheap to pay for any of them. I've only tried gpt-oss:20b because I can run it locally and it's a complete waste of time for anything except code completions.

I really struggle to see the usecase of Grok Code Fast when you have Qwen 3 Coder right there providing much better outputs while still being fast and cheap.

how are you using grok code fast? what tooling/cli/etc?

IIRC sonnet 3.5 (and definitely 3.5-new aka 3.6) was better than opus 3.

There's still a lot of low hanging fruit apparently

That minutiae was always borderline irrelevant, the skill was always making somebody money, possibly with software.

The reality is that more software will be pushed than before, and more of it will need to be overseen by a professional.

That code taste of yours, your ability to understand the minutiae, makes you superhuman at managing agents.

Comments, docstrings, naming, patterns - by defining better approaches and hold agents to them, the results will be better. Way better.

You can't grow a meaningful codebase without solid underlying primitives. The entropy will eat you alive.

Systems architecture is becoming more important - systems that play well with agents wind up looking more like enterprise codebases.

I've been deep into AI full-time professionally for some months now, and for the first 4+ weeks I felt the exact same way as you describe - it is a form of existential crisis, especially after spending the bulk of the past 25 years honing my coding-fu algo ninja skills, my identity was totally wrapped up in it.

Keep at it and keep leaning in to embrace it, I promise it gets better! It's just a big adjustment.

You are FAR more than your code!

Don't be so grim! This will just give you access to not worry about writing clean code as much as you did in the past - you can focus on other parts of the development lifecycle.

The skill of writing good quality code is still going to be beneficial, maybe less emphasized on writing side, but critical of shipping good code, even when someone (something) else wrote it.

You were likely happy to be automating other people out of a job, now it's happening to you. This is the creative destruction that is critical to a healthy and prosperous economy.

I used to think so. But now I think it's not very useful with what I've seen from others. Maybe if you do frontend.. The people who I see vibe coding with no experience actually programming... It is completely useless. It can only do the most simple tasks, anything beyond it will constantly make critical errors and random mistakes. That is using what was the latest Claude version before this. I've also not really used AI coding stuff myself at all do take that as you will.

Even for frontend tasks it makes mistakes when you ask too much of it...

What will it create for me? A basic react/nextjs frontend and a <popular website> clone? Anything that requires more careful planning and esoteric functionality it can't do.

Oh yes, and the code I've seen it write... It can take what should be 20 lines of code and turn it into 600!

But systems level thinking, taste, technical creativity and all other “soft” skills have never been more relevant. I can do some pretty awesome things with my aider. I can implement things which I thought were cool and useful but couldn’t be bothered to without AI.

If anything is depressing, it's the amount of people who think these models actually "can code".

Because they can't.

Also, Clean Code is a really bad ideology, you should regret wasting time on it regardless of LLM code generation.

I’ve been using AI coding tools (Cursor, Claude Code) for React/React Native side projects. I have experience with these frameworks so I could guide the AI with individual tasks and catch mistakes, and overall it worked pretty well.

Recently I tried building a native iOS app with zero Swift experience, giving the AI just a markdown spec. This was basically vibe coding, I didn’t understand much beyond general software principles. It quickly broke down: hallucinated method signatures, got stuck on implementing extensions, and couldn’t recover. I would run the app on my device and give it feedback and logs. After hours wasted, I spent some time reading the docs and fixed the issues myself in 30 minutes.

My takeaway: AI will accelerate developers but won’t replace them. Still, acceleration means fewer engineers will be needed to ship the same amount of work.

Eh, I’ve gotten over that. I’ve been using Claude recently on a personal project for a friend who wanted to take a known export file format and turn it into a list of good households for local political candidates to hit when knocking on doors. And I did that. But it’s been a while since I used pandas and numpy so I told Claude to swap out my loops for efficient code. And he did. Then, just for fun, I said, “Hey, since I am providing you with street lengths and long/ lats, use K means clustering to group high scoring houses into walkable routes and then plot the whole thing in a map from OpenStreetMap.” Five minutes later I had all of that. I could have done the latter, but doin any “real CS” thing would take me days. There’s not a bunch of value in me taking days to do something but there is value in knowing about K means clustering, knowing OpenStreetMap exists and having a feel for efficient code. Plus more high-level things like what good code does and doesn’t look like.

You are not alone

Likewise, a lot of what we learn at school or university is superceded by new knowledge or technology (who needs arithmetic, when we all have a calculator in our pocket??), but having an intimate knowledge of those building blocks is still key to having a deeper and more valuable aptitude in your field.

Trust me, these “vibe coding” tools don’t speed up productivity much in the real world. At the end of the day these systems need to be maintained by humans and humans are the ones held accountable when stuff breaks. That means humans need to understand how the systems (code, infrastructure, etc) work. You can automate the code, even some decisionmaking about how the program should be organized, but you can’t automate the process of having a human develop their mental model of how & why the system works. That was always the bottleneck and still is to this day.

When everyone else has given up on software dev as a career path, you’ll be one of the few who the CEO can call upon to explain why X or Y broke, and fix it. That will make you insanely valuable.

It's not wasted imo. Llms don't get 'it'. They can't tell what's truely good. You have to guide them. To do that you have to know what your doing.

characterize it in terms of truth, clarity of truth, simplicity, and correctness. I think we should always evaluate things along those dimensions. Is it true, does it produce truthful things. It makes the evaluation very objective.

I think clean code is more important than ever. LLMs can work better with good code (no surprise), and they are trained on so much shit code they produce garbage in terms of clean code. They also don't have a good taste or deeper architectural understanding if big codebasis where it's even more important.

What you learned over the years, you can just scale up with agents.

And now wait till you realize it's all built on stolen code written by people like you and me.

GOFAI failed because paying intelligent/competent/capable people enough for their time to implement intelligence by writing all the necessary rules and algorithms was uneconomical.

GenAI solved it by repurposing already performed work, deriving the rules ("weights") from it automatically, thus massively increasing the value of that work, without giving any extra compensation to the workers. Same with art, translations and anything else which can be fed into RL.

[dead]

the investment bubble will burst, at which point the cost to run the plagiarism machine will increase by a factor of 100

Maybe this is true? But it's not clear to me this methodology will ever be quite as good as native tool calling. Or maybe I don't know the benchmark well enough, I just assume it's vision based

Perhaps Tesla FSD is a similar example where in practice self driving with vision should be possible (humans), but is fundamentally harder and more error prone than having better data. It seems to me very error prone and expensive in tokens to use computer screens as a fundamental unit.

But at the same rate, I'm sure there are many tasks which could be automated as well, so shrug

Do you think a Genie like model specifically trained on data consisting of interacting with application interfaces would be good on computer use tasks?

What are some standard benchmarks you look at in this space?

> There's also a bug where compaction becomes impossible. ("conversation too long" and its advice on how to fix doesn't work)

I have seen this issue with every model so far

I'm one of them, but I'm just a product designer who likes to jump between various AI tools to get experience with them. Once my month with OpenAI is up, I may jump back to CC as I liked some of the non-coding features more, specifically plan mode.

Going from pro to Max was a giant let down.

Then they even started sending me marketing emails which was the straw that broke the camel's back, I use to cancel subscriptions of companies spamming my email.

*vw (volkswagen)

I never understood the point of the pellican on a bicycle exercise: LLMs coding agent doesnt have any way to see the output. It means the only thing this test is testing, is the ability of the LLMs to memorise.

Edit: just to show my point, a regular human on a bicycle is way worse with the same model: https://i.imgur.com/flxSJI9.png

pelican on a bicycle benchmark probably getting saturated... especially as it's become a popular way to demonstrate model ability quickly

... but can it create an svg renderer for claude's site.

Anecdotally this new Sonnet model is massively falling apart on my tool call based workflows.

I’m having to handhold it through analysis tasks.

At one point it wrote a python script that took my files it needed to investigate and iterated through them and ran `print(f”{i}. {file}”)` then printed “Ready to investigate files…” And that’s all the script did.

I have no idea what’s going on with those benchmarks if this is real world use.

I think so. Because systems require a lot of knowledge to create and maintain without breaking.

How many years till AI can be trusted to deploy changes to production without supervision?

Maybe never.

Isn't Opus much slower than Sonnet? I haven't been using Opus for that reason

Excited to try Claude agents sdk though

I'm wondering if it's not just: spawn multiple time the same prompt and take the best

It means that the benchmark isn't useful anymore and we need to build a harder one.

edit: as far as what the numbers mean, they are arbitrary. They are only useful insofar as you can run two models (or two versions of the same model) on the same benchmark, and compare the numbers. But on an absolute scale the numbers don't mean anything.

then we need new bench.

This isn't new to other models, and it doesn't make much sense to specify both.

Why might this be, does anyone know?

I'm sympathetic, but it occcured to me that ccording to my amatuer studies, Germany lost WW2 in part because it had a craftsman mentality to manufacture war machines and ended up with a bazillion different part requirments and a shortage of skilled craftsmen, while America used Henry Ford's assembly line process to stamp out hundreds of thousands of identical machines sharing the same parts.

Now we are at the assembly line stage of software production with AI. Us craftsmen will have to find other ways to enjoy our crafts.

Yup, we're headed to the robot assembly line, with a few experts making sure it all works correctly.

Craftsmen will remain, but it will be niche (and probably not pay anything unless you are a true master).

From simonw's blog: https://static.simonwillison.net/static/2025/claude-sonnet-4...

Really? Because this feels like a pretty significant downgrade from Sonnet 4 on real world tasks.

would love to see the prompt they used and the final code of the Claude.ai clone it generated

Possibly, or they are pricing for sustainability and OpenAI/xAI are just burning through VC money.

The Anthropic models are also better at coding. Why wouldn’t they price it higher?

It's meant to be used with the Max subscription

it looks like its already there

The price of training and running the models doesn't really change much no matter which region you're hosting/making requests from.

Regional pricing unfortunately doesn't really make much sense for them unless they're willing to take even larger losses, even if it is a barrier to lower income countries/regions.

Wonder no more: https://github.blog/changelog/2025-09-29-anthropic-claude-so...

I heard 5x cheaper then opus

Pricing is the same as sonnet 4

For now, yeah. Presumably they'll come out with Opus 4.5 soon.

Don't think I've ever preferred Opus to Sonnet

These reviews are pretty useless to other developers. Models perform vastly differently with each language, task type, framework.

Most people here use these models as you can see from the comments.

But we can also see that we're one of the few sane skeptical places in a world that is making the most diverse claims about AI.

I am almost convinced your comment is parody but I am not entirely sure.

You want proof for critical/supportive criticism? Then almost in the same sentence you make an insane claim without backing this up by any evidence.

> include some specifics of the task and prompt. Like actual prompt, actual bugs, actual feature, etc.

> I am still surprised at things it cannot do, for example Claude code could not seem to stitch together three screens in an iOS app using the latest SwiftUI (I am not an iOS dev).

You made a critical comment yet didn't follow your own rules lol.

> it's so helpful for meaningful conversation!

How so?

FWIW - I too have used LLMs for both coding and personal prompting. I think the general conclusion is that it when it works, it works well but when it fails it can fail miserably and be disastrous. I've come to conclusion because I read people complaining here and through my own experience.

Here's the problem:

- It's not valuable for me to print out my whole prompt sequence (and context for that matter) in a message board. The effort is boundless and the return is minimal.

- LLMs should just work(TM). The fact that they can fail so spectacularly is a glaring issue. These aren't just bugs, they are foundational because LLMs by their nature are probabilistic and not deterministic. Which means providing specific defect criteria has limited value.

Could you share the actual examples of where you’re seeing the 3x output increase?

I had a complete shocker with all of Claude, GitHub Copilot, and ChatGPT when trying to prototype an iOS app in Swift around 12 months ago. They would all really struggle to generate anything usable, and making any progress was incredibly slow due to all the problems I was running into.

This was in stark contrast to my experience with TypeScript/NextJS, Python, and C#. Most of the time output quality for these was at least usefully good. Occasionally you’d get stuck in a tarpit of bullshit/hallucination around anything very new that hadn’t been in the training dataset for the model release you were using.

My take: there simply isn’t the community, thought leadership, and sheer volume of content around Swift that there is around these other languages. This means both lower quantity and lower quality of training data for Swift as compared to these other languages.

And that, unfortunately, plays negatively into the quality of LLM output for app development in Swift.

(Anyone who knows better, feel free to shoot me down.)

How do you measure 3x sustained output increase?

Is it number of lines? Tickets closed? PRs opened or merged? Number of happy customers?

HN is such a negative and cynical place these days that it's just not worth it. I just don't have the patience to hear yet another anti-AI rant, or have someone who is ideologically opposed to AI nitpick its output. Like you, I've found AI to be a huge help for my work, and I'm happy to keep outcompeting the people who are too stubborn to approach it with an open mind.

> for example Claude code could not seem to stitch together three screens in an iOS app using the latest SwiftUI

have you tried in the new xcode extension? that tool is surprisingly good in my limited use. one of the few times xcode has impressed me in my 2 yeasrs of use. read some anecdotes that claude in the xcode tool is more accurate than standard claude code for Swift. i havent noticed that myself but only used the xcode tool twice so far

> based on your actual work, include some specifics of the task and prompt.

Can't show prompts and actual, real work, because, well, it's confidential, and I'd like to get a paycheck instead of a court summons sometime in the next two weeks.

Generally, 'I can't show you the details of my work' isn't a barrier in communicating about tech, because you can generalize and strip out the proprietary bits, but because LLM behavior is incredibly idiosyncratic, by the time you do that, you're no longer accurately communicating the problem that you're having.

I had Claude Code build a fairly complex swiftui app (5+ screens), using Firebase AI Logic and other packages. First prompt it got pretty much foundation for the entire thing set up, then over the next day got it working exactly like I wanted. The thing that took the longest was getting through app review. I was impressed how well it knew SwiftUI and Swift composable architecture.

New Claude Model Runs 30-Hour Marathon To Create 11,000-Line Slack Clone

https://www.theverge.com/ai-artificial-intelligence/787524/a...

Yeah, maybe it is garbage. But it is still another milestone, if it can do this, then it probably does ok with the smaller things.

This keeps incrementing from "garbage" to "wow this is amazing" at each new level. We're already forgetting that this was unbelievable magic a couple years ago.

> for example Claude code could not seem to stitch together three screens in an iOS app using the latest SwiftUI

That's... not super surprising? SwiftUI changes pretty dang often, and the knowledge cutoff doesn't progress fast enough to cover every use-case.

I use Claude to write GTK interfaces, which is a UI library with a much slower update cadence. LLMs seem to have a pretty easy time working with bog-standard libraries that don't make giant idiomatic changes.

Would you be so kind to lead by example?

What are the specific tasks + prompts giving you an 3x increased output, and conversely, what tasks don't work at all?

After an admittedly cursory scan of your blog and the repos in your GH account I don't find anything in this direction.

Would you say you do things you'd normally do 3 times faster? Or does it help you move past the things you'd get stuck on or avoid in the past, resulting in an overall 3x speedup?

I agree. I think we can start with cloverich including some specifics of the task and prompt.

this is a great copypasta

Definitely an overall positive with the negatives actually being kind of hilarious and no big deal which I'll also discuss.

I can only list my open source outputs concretely for obvious reasons but https://github.com/rubberduckmaths/reddit_terraforming_mars_... was a near one shot. It's a Reddit bot that posts card text to the Terraforming Mars subreddit when asked which is helpful for context on discussions of that board game. Appreciated and used a lot by the community there. There's a similar project i used AI for to scrape card text that was also near one shot. I'd say for these two hobby projects 50x productivity is a reasonable statement. I wrote Reddit bots ~10 years ago without coding assistance - https://github.com/AReallyGoodName/xwingminibot i get to reasonably absolutely compare two very similar projects. I think it's totally fair for me to say 50x for this example. The Reddit API even changed completely in that time so no one can really say "you used past experience to move faster, it's not the ai giving a 50x boost" but I really didn't. My memory is not that good except for memory of an entire weekend previously vs <30mins total now using a bot to one shot some pretty cool projects.

As for the negatives they are never serious. A couple of good examples;

"Please correct all lint errors in this project" only to have @lintignore added to all files. Lol! Obviously i just more clearly specified the prompt and it's not like it's hard to catch these things and not ship to prod. It was funny to everyone i showed and no big deal.

Another similar case, "please make the logging of this file less verbose especially around the tight loop on line X". Instead of changing log level or removing some of the log statements the ai redirected stdout at the initialization of the command line program (would completely break it of course). Again hilarious but also not big deal. Not even much of a waste of time since you just change the prompt and run again and honestly a few silly diversions like this now and then is kind of fun. As in the comments of "OMG AI sometimes gets it wrong" aren't at all serious. I have version control, i review code. No big deal.

I too eye roll massively at some of the criticisms at this point. It's like people are stretching to claim everyone who's using a coding assistant is newb who's throwing everything into prod and deleting databases etc. That's just not reality.

"Please include ACTUAL EVIDENCE!"

"I tripled my output (I provide no evidence for this claim)"

Never change, HN.

Mouthfeel, as it were

Pervert.

I have personally witnessed Cursor diagnose and then fix an actual non-trivial bug starting from its description in Jira, and I only had to course-correct it once.

I have also seen it fail on far simpler tasks.

It varies so much depending on what you are doing, the language etc that generic proclamations "it works!" or "it doesn't work!" are pretty much meaningless.

That aside, you seem to be conflating "it works" with "good enough to replace programmers", but these aren't synonyms.

And on the gripping hand, one way to "make" it work is simply to lower the standards. Which our industry has been doing aplenty for a long time now even before AI, so we shouldn't be surprised when top management drives it to its logical completion.

Very interesting observation.

I haven’t written a function by hand in 18 months.

I have 20 years of experience and am now just running Opus 4.1 all the time. Of course it’s possible.

In my case I found having it always Ultrathink and to always work with TDD to work well. Also you have to use Plan Mode first and refine the plan. “What clarifying questions do you have for me” prompts me with a handful of numbered questions that are always really beneficial for refining the plan.

every attempt I've attempted thus far has failed at first run, but it was close to a solution that could be adapted and fixed. This has been especially helpful in areas where I'm charting very unfamiliar territory for very narrow scoped problems.

But hearing your 10 line constraint gives me a very https://en.wikipedia.org/wiki/Infinite_monkey_theorem vibe to the challenge.

sounds like poor prompt engineering. Devin and Claude can both do better work than many interns I have mentored in my career, and faster too. We likely have many many years until it will be even close to replacing an experienced developer but we are already at the point it IS replacing junior engineers.

Whether you agree or not, the market has spoken. New grad hiring is WAY down. Fresh CS grads are having an hell of a time finding work compared to 2 years ago.