I figure at the very least I should be able to manage a career as a debugger for superficially-close-but-actually-broken AI programs for a few years, just have to keep my corrections from being used to train the debugger AI

Was playing with it yesterday. Gave it a Clojurescript task between two clients which looked okay'ish even if not exactly what I meant. Then I instructed to make it via websockets and it switched to Python! It's very cool, but I wouldn't start worrying yet. 😅

As coders I think we're always inventing between wheels actually, which is probably unusual compared to other jobs. That is we already know that if a library is available, we'd use it and wouldn't re-invent it. What's implied by that is that a data munching AI would be very good at exactly the kinds of problems you wouldn't need to code because it's there as a library or very tedious coding layout stuff that we wouldn't want to do anyway, and so it would save us time.

But was it good Python and was it according to the task? Because at the end of the day it's not you who'll execute the code :thinking_face:

The real question when money is involved is would I sign my name on it? ;-)

Now ask it to "get that data Bob wanted" which more closely resembles the tasks most devs get asked to do.

I think that you could definitely ask it to perform sub-tasks such that it would print what you’d need to enter into some system that you have access to

I guess I'm trying to say that the question is the answer and the hardest issues i face aren't solving a local minimum (getting an answer to a question regardless of context) but figuring out how to make my steps towards success bigger and in the right direction in an existing context. I think we have all being using AI for years. We call it Google.

The biggest area of improvement becoming available is that you can probably create a Google for your internal company without sending all your ip to another company and hoping they don't steal stuff. Idk. I'm out of my league on this topic, not sure why i chimed in lol.

> But was it good Python and was it according to the task? Because at the end of the day it's not you who'll execute the code :thinking_face: I feel like that’s the core issue I see with these most of the time. E.g. the example in the original post has a syntax issue with the second let and once you fix that it’s still just an infinite loop. Had a similar conversation with someone about Warp in the shell last week where it very quickly gave them a solution that appeared it could be correct, but a one character flag passed to one command gave the answer different semantics than the question.

> one character flag passed to one command gave the answer different semantics than the question I think that this has the same core problem as using Copilot. When code works everyone is happy (if it works). But when you need to understand why it works and who to give attribution to then you'll at least need scratch your head. And then we arrive to this same conclusion from the beginning of the discussion: > would I sign my name on it?

> I asked somebody to do something and when it became too hard they switched to python to do it This actually makes the AI more like a human, if anything 😄

I have definitely seen people do that

What if it gets too out of hand? Is it reasonable to expect that it'd switch to spreadsheets? xD

So, i'm trying out chatGPT, and at first i'm like wow. But then i realized, it's just making things up when it doesn't know the answer. That or i'm missing things in the docs. Its like use the function-with-the-name-you-asked-for to do the thing you asked for.

Which makes sense if they trained it on software engineers. 💪

it's got a lot of knowledge and absolutely no shame about BSing its way through an answer

Actually, i'm still in awe. > those functions dont exist in the cider docs, are you just making them up? > My apologies if I gave incorrect information. The cider-refresh-dynamic-completions and cider-refresh-dynamic-completions-in-current-ns commands do not exist in Cider. These commands were part of an older version of Cider, but they have since been removed. > To refresh Cider's auto-completions in the current namespace, you can use the cider-repl-clear-buffer command, which will clear the REPL buffer and re-evaluate the code in the current namespace. This will update Cider's completions table with any new functions, variables, or other symbols that are defined. > To refresh the auto-completions for the entire project, you can use the cider-ns-refresh command, which will re-evaluate all of the code in your project and update Cider's completions table with any new symbols that are defined. > I apologize for any confusion or inconvenience caused by my previous response. I am a large language model trained by OpenAI, and my answers are based on the information available to me at the time of the query. My knowledge is limited to what I have been trained on, and I may not have all of the correct or up-to-date information. If you have any further questions or concerns, please don't hesitate to ask.

you'd have to actually go verify if that's factual or not though

I didn't save it, but I was playing with it this weekend asking it to design SQL schema. At one point I asked it to setup row-level security using some user/role tables defined previously and it came back with something nonsensical using a valid keyword in the totally wrong context. I asked how confident it was that its solution was correct, and it was like oh 100% boss. I asked it to update the solution to use a particular postgresql version and it told me a bunch of things about postgresql and then gave me another broken schema.

It's just covering for itself 😉

Damn good try, though.

i'm in awe of it's commitment to the lie then. The next step is that it actually writes the function and alters the git history.

to its credit, I asked it to give me examples of using the same schema with datomic and it immediately switched contexts and gave me a bunch of examples (that were probably all incorrect somehow) that referenced all of the various fields we'd discussed and domain-level operations I was interested in modeling

yeah, this is super useful. Like, i feel like i knew this was coming but seeing it first hand is still a shock.

it does feel like there's something to the nature of the stateful interactions, if the interface could just express a confidence level somehow

> it does feel like there's something to the nature of the stateful interactions, if the interface could just express a confidence level somehow My guess, although I could certainly be wrong, is that rather than being truly stateful, it's using the whole history (or the history back n words) as the prompt, since that's a really easy way to get GPT to be context-aware.

I realize that's nitpicking though 😉

yeah, I wouldn't be surprised if it's that or essentially that. It's just disarmingly convincing as soon as you get a little beyond your own experience

If you want to craft a narrative that plays ball the same way as the message your klingons do, then start by telling them that you don't want to use typescript because it's a none algorithmic language and you like the power and safety a data structure first language like Clojure affords you.

They will ask you what you mean, and you will get to explain more.

In the end they might end up knowing clojure better then typescript, and then you have new friends you can talk to in the same language.

This was generated by ChatGPT based on the cited prompt. I just thought it might be fun to share here. 😅

I mean, weird flex but ok.

That's a pretty funny conversation 🙂 Thanks for sharing 😉