The training sets typically includes big codebases downloaded from GitHub in addition to small snippets from places like stackoverflow. One key limitation preventing this is the context window size. Early versions of GPT-3 only could only look backwards 2,000 to 4,000 tokens (which is approx 1500 to 3000 words). This means after the AI has generated 1500 to 3000 words of output it can only see what it has produced and not what came before it (ie the prompt from the human). It may not even be able to see the namespace name or :require statements! This limitation can be fixed/improved over time with non-fixed context windows or larger fixed context windows or some other approach. This is one reason why tools like co-pilot can struggle recommending/utilising existing functions/libraries within a large codebase - there is simply no way to feed in all relevant functions/libraries into the prompt that may be needed. Instead CoPilot in a large codebase often resorts to suggesting open source libraries (seen during training), re-inventing the wheel from scratch or hallucinating new/made up functions.

yes looks like that, that it has problems combining the context that already produced, thank you : ) i hope AI to be used in problems that we dont have people to solve them, else it will start replacing people, especially knowledge based jobs.

I think if you read this, then you can get a pretty good ide of what ChatGPT is doing, and thus what its limits are; https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

It seems to me (and I’m far from knowledgable) that combination of a GPT-3 approach and other approaches will be what will allow AI to generate large and cohesive codebases.

It would take several leaps still because GPT certainly can't produce large codebases yet. • Current GPT models often produce broken and syntactically incorrect code. GPT models can produce a plausible next token - but hasn't got a true understanding of how the code runs (or even the syntax of it). ◦ To prove this: ▪︎ Each execution of the GPT Neural network takes a fixed amount of time - but to mentally compile/evaluate code with loops, conditionals and recursion takes a non fixed amount of time. So it is clear that the GPT model is not mentally evaluating/compiling the code - again it's just producing plausible new tokens given what it has seen/outputted so far. • GPT doesn't back track or rewrite code - there may be hard to forsee issues with an approach that can only be known multiple steps down the line (e.g. multiple files/lines of code later) - but once GPT has outputted a token - it never changes it's mind. • The well known GPT hallucination problem also applies to codebases. Whilst generating code GPT may get into a tricky situation and so it will hallucinate in new library functions that don't exist or even hallucinate new features of the programming language to dig it's way out.

I’ve definitely seen the ’hallucination”. What’s interesting is that the hallucination sometimes seems to suggest what should naturally be there (but isn’t in real life).

Thank you for such a complete and well organized response, btw!

I think the down-the-road scenario will be when GPT (or something else) does backtrack and rewrite code.

Yes the hallucinations could be used to create the functions that it hallucinates - but this may just hallucinate even more functions within these.

That more or less describes my coding process. 😀

I can confirm that he hallucinate often. And one of them was ruining my entire PLT career. We shouldn’t see it as a source of truth or knowledge, we should see it as a developed version of what we tried to find in Google or some tools.