Fork me on GitHub

Thank you so much @rupert: this is very helpful. The only issues (1) I am thinking of now is the size of the dowloaded data (pdfs or webpages if there is huge number of them) and (2) on the intensity of training NLP. I am assuming for (1), saving to cloud service storage is one way solve it, and for (2) use GPU service, in which case the date needs to be saved on their database? thanks again.

Rupert (All Street)10:01:24

If you are just getting started, then you probably want to develop your algorithm end to end running on your local computer (with a small subset of data) - that way you have a fast feedback loop. As this is is a data science task you will spend a lot of time looking at data, optimising the AI performance and not just writing code. Once it's all running nicely then you can scale it up to run on clouds etc. When you have parsed out the text and from PDF/HTML and compressed it, it is likely to be approx 1% of the original data size so you can download data in batches, extract then delete the original files to save space. For Model training you can buy a used 1080 TI on ebay for $200 (USD). It might be slower than the latest Nvidia GPUs, but it just means that you wait a little longer for training to complete. Often training takes hours anyway - people often do it overnight. If you want to use a remote GPU, the cheapest are on . If you use a GPU on Cloud servers then you pay per second and you need to be constantly stopping your server when idle to not run up very large bills. If you are new to NLP I would focus on that bit first (and leave out all the cloud storage, databases, distributed compute architecture until after you have that working).


Makes sense. My goal is to use a pretrained model such as chatgpt on a large text data sent (from pdf or webpages)

Rupert (All Street)16:01:39

If you are using ChatGPT then you won't need a GPU - since it's a managed service in the cloud. There's no AI training because it's a prebuilt model, so your NLP task will either be prompt writing or fine tuning (if you really need to). You will likely have to keep your data volumes pretty low going by their existing

šŸ‘ 2

But I need to feed it the large dataset though.

Rupert (All Street)18:01:03

Sure - you can calculate up front what it will ā€¢ Davinici model is $0.02 per 1,000 tokens. Each word is about 1.33 tokens. ā€¢ So 1 MB of text (which is about 209,000 words or 280,000 tokens) would cost $5.20 USD. ā€¢ 1 GB of text would cost $5,700 USD You can reduce the cost by processing less text or using a less accurate model. Alternatively use a different model and/or different provider. Open AI is the most well known but its not the only one.

šŸ‘ 2
Carsten Behring21:01:05

What do you want to achieve? ChatGPT is mainly usefull for dialogs, not for giving it large texts.


I would like to use ChatGPT API to train on a text to create a chatbot

Rupert (All Street)14:01:17

I think it may help if you are more specific about your requirements or your idea. I'm not sure that ChatGPT will do what you are after. For example ChatGPT does not give back accurate or factual information - even with additional training. ChatGPT is already trained on millions of web pages and PDFs - it might already have trained on your material which means you may not need to download them in the first place. Also you need to figure out your budge because Open AI's APIs cost money as I covered in my last message.

šŸ‘ 2

I have a specific text (an in house, non-piblic document), I would like a chatbot answering any question regarding the policy...maybe this does not make too much, but I am a hobyist trying fun project to learn.

Rupert (All Street)20:01:10

Sounds like a fun project. I would try testing the idea with Chat GPT in their Web UI (by copying bits of policy and asking questions) etc. If you think it's working well then you can productionise it with code/API etc.

šŸ‘ 2

I did not think of that. Will definitely try

Carsten Behring17:01:39

ChatGPT is not made for "Giving it a long text and ask questions on the text". It is made for asking it questions "about the world". (and it has learned those in a one time training from massive amount of news articles) You can give it some context as part of the questions, but it will always look for answer elsewhere as well (in its original training data)

šŸ‘ 4

do you know of free NLP pre-trained models that I can use on my text data for the task that you would recommend?

Rupert (All Street)09:01:13

There's which has nlp libraries including gpt-2 that you can run locally. It's Python so you can either(A) use it from lib-pythonclj or (B) have clojure read/write JSON/CSV files to interact with python code. Depending on the model you choose, you will likely need an NVidia GPU with at least 8GB of GPU memory.

šŸ‘ 2
Carsten Behring11:01:12

Your 'task' is called So you should pick a model which is optimized for it

šŸ‘ 2

Excellent, thank you all.