Fork me on GitHub
#off-topic
<
2022-09-02
>
Jan ariane17:09:13

is there any other better way to store data except spreadsheet (2D-dimensional) so that data finding+extraction becomes easy & fast later

Daniel Craig17:09:36

Databases are my go-to for something like this

Daniel Craig17:09:16

So many database options but they all accomplish the goal of making data finding/extraction easy and fast

p-himik17:09:17

Not sure what you men. Store - how? In memory or on disk? Finding - how? By the 2D coordinates or by values? "Later" - why not now? "Spreadsheet" - is there any other reason to mention it, apart from it being a 2D thing (for which saying "a 2D array" would suffice)?

Daniel Craig17:09:21

I'm not sure what a good starting place is here, since my clojure + database experience only includes graph databases which aren't really the 2D type of thing you're looking for

Daniel Craig17:09:40

Someone else can weigh in

Jan ariane17:09:53

in fact i would prefer n-D

Jan ariane17:09:17

2D is what our eyes can see & easy to type in data, right?

Jan ariane17:09:37

• store= type or manually feed • i have all cases i.e. memory, disk & both • by later, i mean after data is created/typed in/stored in • spreadsheet i guess is better than markdown since it can allow 2 dimensional stuffs to store easily. of course, for linear data/indented data, markdown/emacs/text are good enough. but not sure how data filtering-extraction-finding works there. • at the end, i need something as elegant/handy as https://espanso.org/ (it can be fired from any app, just by a shortcut & then extracted data can be pasted in any external app e.g. email)

Daniel Craig17:09:24

OK have you used a database before?

Daniel Craig17:09:46

no shame in being new to databases

Jan ariane17:09:49

i had used power query in excel it felt like a database

Jan ariane17:09:58

nothing else than that

Daniel Craig17:09:45

So the most popular databases use SQL, it's a query language the makes extracting data fast and easy

Daniel Craig17:09:26

So for a free database that uses SQL, many folks use Postgres

Daniel Craig17:09:22

In database systems built around SQL, data is stored in tables which have rows and columns, so that would be familiar to you

1
Daniel Craig17:09:33

Lots to learn here

Daniel Craig17:09:02

I would recommend you search for a video tutorial series on SQL. You could use Postgres, MySQL, or SQLite, and don't be afraid to take the time to really learn the tool. Knowledge of databases will take you to the next level in your software journey

👍 1
pppaul19:09:01

you could look into apache arrow

pppaul19:09:37

which is a type of database, but it's more similar to a spreadsheet than a DB

pppaul19:09:58

a typical SQL db requires a lot of work to get up and running and integrate with your data (depending on your data)

pppaul19:09:11

there are solutions in the data-science space that are DBish but for different use-cases and typically with less upfront cost (like apache arrow)

pppaul19:09:06

if you have no foundation in DB stuff at all, then something like https://sqliteonline.com/ + videos like daniel recommended are probably good. i would recommend staying a bit far from ORM stuff, and use something like yesSQL in your app (maybe there is a newer thing that is like that now)

pppaul19:09:11

but expect to go down a bit of a rabbit hole, SQL is about as hard as learning clojure.

p-himik19:09:34

To add to the above - just extracting the data at its coordinates in SQL is about as simple as get-in in Clojure. ;)

pppaul19:09:37

though, i would also say it's about as hard as learning excel too, so if you are good at that then you are probably ok

pppaul19:09:43

SQL dbs do have data importing features, so you may want to look into those. you can probably take your csv and put it in the DB with a command line util

👍 1
Jan ariane19:09:53

i need suggestions for affordable (for international students) schools/colleges for computer science/AI/deep-machine learning or software or similar.

Jan ariane19:09:17

what do you recommend/use instead of Notion ?

sheluchin20:09:50

https://logseq.com/ is pretty popular and written in Clojure.

👍 3
Jan ariane20:09:52

i think tabular support lacks in it

isak20:09:00

I'm also interested. I have seen https://www.nocodb.com/, but have not tested it yet.

Jan ariane03:09:22

you may keep an eye on http://AFFINE.pro too. Microsoft List is also nice option.

paulocuneo05:09:56

SQL editors/IDE used to be so bad, that now people are building no-code platforms to make use databases 🙃

Ben Sless04:09:01

Table support in logseq is meh but you can get a plugin for that https://github.com/haydenull/logseq-plugin-markdown-table