Fork me on GitHub
#clojure-europe
<
2021-03-03
>
kardan07:03:46

Good morning

agigao08:03:23

დილა მშვიდობის!

ordnungswidrig09:03:32

@chokheli I realize that the georgian script (mchedrull, right?) is liked by a lot of people. I wonder if there are different variations like sans-serif or like that. To me it seams like there is only that single “rounded one”.

javahippie09:03:12

Something I never really thought about when choosing fonts, that some alphabets are not supported.

RAMart11:03:44

And be careful when choosing, publishing and redistributing fonts, because fonts have licenses too.

👍 6
ordnungswidrig13:03:52

@javahippie that’s an intersting collection. It seems to me that the western scripts have a wider variety in styles. But that might also be because computers and electronic fonts clearly a are biased to western scripts.

ordnungswidrig13:03:00

(Or even just ASCII)

javahippie13:03:00

If you type some letters from the georgian alpabet into google fonts, you have a really hard time finding any that work. I also guess that this is because most of IT is western-centric

javahippie13:03:22

Even Noto Sans only works in bold:

agigao17:03:29

@U054UD60U Mkhedruli is exactly the rounded one, some variations of fonts fs, but that’s a standard. On the other hand, previous “version” of Georgian alphabets were quite different, more squared - https://ka.wikipedia.org/wiki/ქართული_დამწერლობა#ანბანი

agigao17:03:44

First 2 rows

ordnungswidrig19:03:17

Oh nice. Ther german page is missing that kind of comparison. But is this used like a font (“Times” vs. “Helvetica”)?

agigao09:03:37

No, only Georgian linguists know the old ones I think.

jkxyz09:03:42

'Neața!

javahippie13:03:56

For a thing I am writing, I’d like to persist EDN to a database in a way, that I can search for (and index) certain values. Time series DB is a plus. Managed to avoid this topic until now, do you have any pointers and experiences? Needs to be open source, as the software I am writing will be, too. The first thing coming to mind for me is datahike.

borkdude13:03:19

@javahippie I would go with datalevin probably

borkdude13:03:50

But that doesn't support history, if you need that, datahike probably works. Datalevin works with GraalVM which is a plus for me :)

javahippie13:03:12

Interesting, didn’t know Datalevin. Also I see loom graph protocols in the roadmap, that’s great as I use loom, From first skimming their page, it seems like it needs lmdb as a backend technology, right? That’s not too common

javahippie13:03:40

Also still under development, and having a clusterable database underneath would be important. But still interesting, will definitely take a look at it!

borkdude13:03:16

Yes, right now you have to install lmdb separately but this will be addressed

borkdude13:03:33

(using automatic bundling of the driver, just like xerial sqlite jdbc does)

borkdude13:03:43

Soon coming as babashka pod too btw

😁 3
borkdude13:03:52

The sqlite that Clojure always wanted, that's how I see it ;)

javahippie13:03:16

Definitely a cool project!

mccraigmccraig13:03:44

i had my hopes up for a moment there that i could add a datalog db to our mobile app... i don't think the clojurey bit will work though

mkvlr13:03:21

will datalevin & asami eventually support similar use cases? Datalevin is getting loom graph protocols https://github.com/juji-io/datalevin#earth_americas-roadmap and asami durable storage https://github.com/threatgrid/asami/compare/storage

borkdude13:03:49

@mkvlr datalevin already supports durable storage

mkvlr13:03:15

@borkdude yep, so coming at it from different directions

borkdude13:03:01

ah I see, I've never looked into the loom graph protocol stuff

ordnungswidrig14:03:52

what about crux on a local rocksd (or lmdb)?

simongray14:03:06

The ongoing friendly competition between Asami, Datalevin and Datahike really excites me, but I wish I could fast-forward about 1 year to know which one to pick for new projects.

reefersleep19:03:18

Dang, I wasn’t even aware of all of these cool things. Sucks that Datomic didn’t work out for us back then. I’m happy to be back in SQL land, and I wonder (after Cassandra) whether some basic (for me) SQL stuff is missing from these alternatives.

reefersleep19:03:25

Like it was in Cassandra, ugh

reefersleep19:03:36

I mean, their briefs are all good

simongray08:03:31

I’ve had to learn a lot about RDF and the Semantic Web at work and could see some clear parallels with the Datalog scene in Clojure, so I documented what I could find here: https://github.com/simongray/clojure-graph-resources#datalog

simongray08:03:48

BTW Søren, you might be interested in this https://www.youtube.com/watch?v=zoOXCaZ3M2Y

reefersleep08:03:21

I am 🙂 Saw it on reddit. Haven’t had time to see it yet

reefersleep08:03:04

Hard to distribute what little time I have among interesting things, there are so many. “Do I really need to see another video about rules engines? I already saw one and grasped the overall concept. Will this one bring anything new to the table?” I’m constantly making decisions like this, and mostly choosing the boring, uninformative and most importantly, time-saving option. Parent life, eh!

simongray09:03:43

It’s only 30 minutes 😉

reefersleep09:03:13

I see that you have an easy child 😛

javahippie14:03:28

Crux is also a good candidate! to be honest, the architecture diagrams on their site scare me 😅

borkdude14:03:06

FWIW Datalevin is used in production within Juji (company) and Asami is used within Cisco

simongray14:03:13

@mkvlr not sure the directions are so different. AFAIK Asami, Datalevin and Datahike are all forks of Datascript.

mpenet14:03:38

they are all sightly different too

mpenet14:03:03

they are not strictly competitors from what I understand

joelkuiper14:03:17

I've been using datahike for a while now, and it seems to work as promised (history support was important to me), the datascript/datalog query language is really quite interesting!

mpenet14:03:23

some support as-of, some don't some have persistence, some dont etc

borkdude14:03:23

That language is directly based on Datomic's query language which has been around at least since 2013 (which is when I first used it) and that is based on Datalog in turn.

joelkuiper14:03:02

which is in turn a subset of prolog, which has been around for a very long time 😛

borkdude14:03:20

> and that is based on Datalog in turn. that's what I said ;)

borkdude14:03:30

sorry, bad reading

borkdude14:03:45

yeah, and prolog is based on.... what's the source of everything?

joelkuiper14:03:55

Horn clauses 😛 ?

joelkuiper14:03:45

alternatively you can also use datascript itself and read/write the edn in a serialization format of your choosing with a watcher as a form of persistence, but it really depends on the size of the data if that makes sense

borkdude14:03:27

You could also use postgres + jsonb :P

ordnungswidrig14:03:37

That works better than I want to confess actually.

borkdude14:03:37

yes, it's pretty good

reefersleep19:03:36

How deeply nested data structures have you worked with (and been happy about)?

ordnungswidrig08:03:12

I don’t mind too much about depth as more about the connectedness (how interconnected a graph is)

reefersleep08:03:47

Just asking because we’ve got some fairly deep structures at work, and they seem a hassle to deal with in the query language - and they’re heavy to query as well, so ideally, you’d put indices in a lot of places in those structures, I guess. I was thinking that maybe it’s a dream for lighter structures.

borkdude09:03:23

I guess that's one of the benefits of triples: no nested structures, easy to query

reefersleep10:03:10

Indeed! I’ve had map fatigue, it’s very real, and I really liked the O’Doyle presentation that @U4P4NREBY linked to in the Clojure Reddit recently for opening my eyes to triples. (I wasn’t mature enough to see the light when I used Datomic early in my Clojure carreer 🙂 )

simongray11:03:11

I think Domain Modeling with Datalog (https://www.youtube.com/watch?v=oo-7mN9WXTw) is what really sparked my interest.

simongray11:03:21

I think the simplicity of modeling using tuples as well as the ability to apply both graph theory and logic programming is what is drawing me to it. SQL is fine and a known quantity, but it’s not without its flaws.

simongray11:03:35

It’s hard to move SQL out of the backend database, whereas the Clojure Datalog paradigm is much more universally applicably and portable.

ordnungswidrig13:03:26

I like the idea of datalog in the frontend.... Where it makes sense.

ordnungswidrig13:03:18

Also change based communication with the backend allows for a nice synchronisation story and even concurrent editing.

ordnungswidrig13:03:27

I mean when you source in events from the server you can then build your local projected datastructures in the way you need them best to query them. Like CQRS does on the server side.

simongray13:03:39

Yup, something like that is the dream. I feel that there is a lot of momentum in the Clojure ecosystem towards creating that kind of thing.

simongray13:03:20

like the whole Fulcro framework, but also lots of smaller libraries. And if you can decompose your data into EAV tuples, you then get to recompose it with lots of things.

javahippie14:03:32

What I like about Crux and Datahike is, that you are able to plug a JDBC datasource for persistence. The software is intended to be hosted on site, and choosing your own persistence provider, even one that supports JDBC and is clusterable on its own would be great

borkdude14:03:50

I think datahike now also supports postgres as storage, if I'm not mistaking

mpenet14:03:31

yes, jdbc, the whole storage story is abstracted so in theory it's quite easy to use on any k-v store

mpenet14:03:04

asami is about to have storage too

borkdude14:03:57

I've heard some performance concerns about datahike vs crux btw, also good to keep in mind

borkdude14:03:06

(crux being the more performant one)

mpenet14:03:07

not sure datahike is optimised (for the write path at least) yet, that will come eventually

joelkuiper14:03:10

I'll second that from experience, the datahike transactions are very slow in comparison with both the leveldb and the file based backend

joelkuiper14:03:19

queries are generally very fast though

mpenet14:03:23

last time I checked (a while ago) there was no batching of writes

borkdude14:03:27

I probably heard that from you then ;)

mpenet14:03:08

crux for writes you essentially benchmark kafka, so yeah it's fast, read-after-write on both would be more interesting

mpenet14:03:40

both are really cool projects, and quite different too again. I am not sure raw performance is a good metric to compare them tbh

borkdude14:03:49

it depends what kind of perf is important for your use case

mpenet14:03:13

depends on what you need to do with the data

joelkuiper14:03:33

the nice thing is that they're pretty much drop in replacements of each other from the query/transaction perspective, so as the projects mature more you can try all of them with relatively little effort

👍 3
joelkuiper14:03:11

well, the core features I guess, history/loom protocols/replication etc will probably remain project dependent

mpenet14:03:14

crux is also likely way more mature

mpenet14:03:34

(than datahike)

otfrom14:03:35

I thought datahike had been around longer

mpenet14:03:49

crux has people working full time on it I think

ordnungswidrig14:03:52

nice, this is from the datalevin readme: > If you are interested in using the dialect of Datalog pioneered by Datomic®, here are your current options: > If you need time travel and rich features backed by the authors of Clojure, you should use https://www.datomic.com. > If you need an in-memory store that has almost the same API as Datomic®, https://github.com/tonsky/datascript is for you. > If you need an in-memory graph database, https://github.com/threatgrid/asami is fast. > If you need features such as bi-temporal graph queries, you may try https://github.com/juxt/crux. > If you need a durable store with some storage choices, you may try https://github.com/replikativ/datahike. > There was also https://github.com/Workiva/eva/, a distributed store, but it is no longer in active development. > If you need a simple and fast durable store with a battle tested backend, give https://github.com/juji-io/datalevin a try.

👍 15
mpenet14:03:20

asami has also as-of iirc

otfrom14:03:06

what I like about crux is that it solves a problem I have a lot in financial services and other things where the application date is more important than the transaction insertion date

mpenet14:03:03

I wish datomic (on-prem) was oss, it's really impressive and I really think it would be a huge boost to clj.

👍 3
mpenet14:03:49

I hope it's in the plans for nubank to do just that, but maybe I am a bit naive

otfrom14:03:29

yeah, I'd be happier to have it fl/oss and pay for support. Had too many proprietary dbs disappear

otfrom14:03:11

looking at hitchhiker-tree looks like a useful thing for me. I do lots of sort these things by date under and id and then reduce the events

otfrom14:03:01

tho some of the work with arrow, clojure and memory mapping being done by the http://tech.ml crew looks interesting too for different sizes of data

javahippie15:03:36

So I have many things to look into now, thanks 😅

otfrom15:03:07

mostly nowadays I use excel and csv for io

otfrom15:03:20

but my data is pretty small

otfrom15:03:09

I do store the raw simulations as compressed transit. I think I might go over to arrow and http://tech.ml.dataset for it in the future though https://github.com/techascent/tech.ml.dataset

ordnungswidrig15:03:11

@otfrom excel local files or excel stored on sharepoint/onedrive/that-office-online-thing?

otfrom15:03:11

well, they start off on my local drive. They end up all sorts of places.

javahippie15:03:30

Wonder if this works with Excel on S3 Buckets https://www.cdata.com/drivers/excel/jdbc/

mccraigmccraig15:03:54

excel makes me 😿 - it has been the source of so many data issues with it's dubious habit of silently changing values in CSV files

ordnungswidrig15:03:00

@mccraigmccraig isn’t the more likely the problem with the understandardized CSV?

otfrom15:03:44

@mccraigmccraig I've started working more w/excel directly to avoid some of the csv export issues

mccraigmccraig15:03:28

yeah, if csv specified type information, then the issues would go away - but they would also go away if excel didn't make type assumptions and convert values into an assumed type. either way it's painful

otfrom16:03:44

yeah, all the source data I deal with is in Excel, so mostly I'm trying to limit the number of changes they are making

ordnungswidrig19:03:17

Isn’t the something in the context of scientific data, R or the like which is a robust data format more flexible than CSV but not an “application format” like Excel? (No, not JSON)

javahippie14:03:49

My wife works as a statistician, and most of what they are using is CSV / other text formats