Fork me on GitHub
#data-science
<
2023-10-10
>
Mattias12:10:46

Hi! Forgive a newbie question, please disregard if I’m completely out of my depth. I’m trying to escape the “Excel for everything” black hole and I’m just trying to learn dataset (& various friends). So, just a starter question, happy for any pointers 🙂 In the docs (at http://techascent.github.io/tech.ml.dataset/), there are references to two namespaces I’m interested in, tech.v3.datatype.functional and tech.v3.datatype.datetime . Can’t find the docs for those, and so I just wonder if there is another place for documentation, do I read the source, or am I confused in some other way?

Mattias13:10:51

:hugging_face:

Mattias14:10:17

Follow up… what’s the easiest or idiomatic way of comparing (filtering) on dates? Must be a common use case 😳 🙂 I have :local-date-time columns 🙂

Mattias14:10:51

For this specific case it’s data I’m happy to filter out when ingesting, if possible. Is that possible?

chrisn14:10:42

filter-column - they should be java time localdates

chrisn14:10:03

tick is the recommended pathway

Rupert (All Street)15:10:35

What are you trying to achieve? You may be able to get by with processing excel/CSV files from clojure and using the clojure.core functions which are easy to learn etc. This isn't like Python where you need a library like Pandas. Clojure is already great at data processing.

Mattias16:10:10

Thanks all. And @UJVEQPAKS, yeah, that’s definately a grey area where, as a newbie, drawing the lines is definately tricky. I’m doing this explicitly to get away from “Excel for everything”, and with time maybe having my colleagues working with Clerk notebooks, etc. More thinking do be done 🙂

chrisn14:10:53

@U0516053R - we are excited we have been looking for something like that for a while. Way back perhaps more than a year ago you can find zulip musings about how to add an SQL engine on top of a collection of TMD datasets or sequences of datasets. This was before we took the most recent look at duckDB and it just feels like a major missing piece of the puzzle.

geraldodev15:10:23

I was talking to my friend earlier today "it's been a long time since I've seen something so useful"