Fork me on GitHub
#off-topic
<
2019-04-11
>
akiroz02:04:26

@hmaurer Wow thanks, that looks really interesting 😄

hmaurer04:04:55

It’s great! I think the best part for me was how much I learned about PostgreSQL’s capabilities by using it. You seem to know them well already from your above comment, but those were mostly unknown to me at the time. Whether or not it is a good idea to build a whole application with a database as the sole backend, I think it’s definitely a fun process. Databases like postgres also do some things really well that are traditionally rather hard to achieve in applications. Access control is one of those things; Postgres is very declarative about it.

seancorfield02:04:49

Windows folks... is anyone else testing the dev or canary channel versions of the new Chromium-based Microsoft Edge browser? Thoughts so far?

borkdude06:04:22

I need to check in a couple of generated JSON files into git. There are data, not code. Does git lfs make sense here for an OSS project? Or are there any disadvantages? Maybe just check them in as normal files?

Lennart Buit06:04:48

I guess that depends on how large they are/how much you expect them to change

Lennart Buit06:04:45

and whether you care about the diff ^^

borkdude06:04:11

about 86kb of json

borkdude06:04:41

changes: don’t know, but I don’t expect them to change more than once every month or so

Lennart Buit06:04:03

well the worst thing in Git is having binaries in your repo because every single version is kept (makes checkout slow), since this is JSON, and frankly it isn’t that large, I guess putting it in the repo will be alright ^^. Reading up on Git LFS, it seems that your contributors need to have it installed as well

borkdude06:04:43

yeah, that’s what I’m worried about… inflicting some tech on other that’s not generally well established

borkdude06:04:55

also I wonder how it will work together with tools.deps

borkdude06:04:03

if you’re using a git dep

Lennart Buit06:04:29

ya, and a 86kb json, I guess stock Git will track that just fine

Lennart Buit06:04:09

(my opinion obviously!)

borkdude06:04:27

yeah, but I tend to agree 😉

dominicm06:04:22

Tdeps works well for this, it only makes one clone of every repo

Lennart Buit06:04:49

(does it make a clone of depth 1?)

borkdude06:04:42

@U09LZR36F “with this” is with vanilla git, or lfs?

dominicm07:04:58

Vanilla git

dominicm07:04:46

@UDF11HLKC I don't believe it is shallow, because that would make it harder to checkout arbitrary shas.

Lennart Buit07:04:36

ah right, that makes sense

victorb12:04:55

Vanilla Git absolutely handles 86kb files without any problems. git lfs is really for binary files that changes often

borkdude12:04:50

yeah. the only objection I would have is that it’s a bit ugly to have data, not code, in git. but I guess it’s fine

dominicm08:04:16

https://martinfowler.com/articles/dblogic.html this seems really interesting. It's sometimes hard to follow the full expected behaviour of something, because the business expectations are scattered across several layers. The flipside is that SQL is faster than doing a filter, especially if you are loading all your data into memory first! I feel like clojure encourages modelling data, then functions on that data, then figure out how to pass data in. But I don't see that so much in practice.

orestis08:04:28

I had a similar issue at work recently (based on Mongo, but still the same principles). Writing Clojure code that just fetches all necessary data from mongo then does the calculations and aggregations manually was trivial, easy to test, reason etc. Writing the same functionality but using Mongo’s “pipelines” took a month. Performance is better with Mongo, but not so much better.

borkdude14:04:00

does anyone have an example how to implement a Doclet (javadoc) in Clojure?

borkdude14:04:34

found something

hipster coder16:04:38

has anyone tried vert.x modular framework on Java. They are working on adding Clojure support. You can run a dozen different languages inside vert.x It looks incredible.

gklijs16:04:46

I have production experience. I think it can be amazing. But in our case it's creating a lot more problems than it solves (with java). And we don't have a real reactive applications, since we don't use vert.x in the front-end. Also the java code is a mess because it's not so functional. But I did a little poc vert.x with a more functional java style, and that worked great.

hipster coder16:04:09

oops.. thanks for threading… I forgot to ask for it.

gklijs16:04:29

If you want high performance you also really need to tweak the memory

hipster coder16:04:53

is the Java code turning into spaghetti because of the async nature?

gklijs16:04:13

Just a small piece of code, as prove of concept on how it could work, using advent of code. Still haven't shown it to the colleagues. They kind of really hate vertX and are building a new piece with spring boot.

hipster coder16:04:05

it says that separating the code…. using the event bus versus the workers can be a mess?

hipster coder16:04:22

the async code (event loop) versus the backend threading worker stuff

gklijs16:04:37

We are now calling it mud rather then spagetti. But introducing async and lazyness to java has some complications if not set op properly. I would really like to see pure functions + endpoints on the database + some routing. I think it could work, but not it takes some discipline. Spring boot is much safer/easier in that sense.

hipster coder16:04:23

if you are open to it… you could github your poc? the more functional style

hipster coder16:04:51

I guess Vert.x seemed attractive because its also easier to setup than Spring

hipster coder16:04:48

I followed you and starred it. Do I need to setup a maven project to use your repo?

hipster coder16:04:05

I see the pom.xml file

gklijs16:04:06

not sure if it's easier, it was a 'new' thing when they started the shared library, and it looks like they are coming back from it. Code for production runs different than local, we've had several issues with that, like missing dependencies.

hipster coder16:04:38

dependency problems with maven?

gklijs16:04:23

mvn exec:java -P vertx should work I guess, it's a small front-end to do the advent of code 2018 questions

hipster coder16:04:34

ok, let me try it, now

gklijs16:04:07

Not really to do with maven, but with how depencies are handled by the verticles. Locally yoiu just have 1 jvm, but on production multiple, and if the scope is set inccorectly for transitive dependencies the verticle will not load them.

hipster coder16:04:55

java.lang.ClassNotFoundException: com.gklijs.adventofcode.MainVerticle

hipster coder16:04:21

I did two steps… 1. brew install maven 2. cd in your gerard dir… 3. run your mvn command

hipster coder16:04:43

I was hoping the mvn command started a vert x server on port 8080

dominicm16:04:10

How does vert support multiple languages?

gklijs16:04:17

It supports (de)serialisation of the vertx messages in different languages

dominicm16:04:52

I see, so it is a worker queue with a java processor?

gklijs16:04:53

Likely you need mvn clean install first

gklijs16:04:47

It's multiple things, it both is something like an actor model, spinning multiple jvm depending on the configuration, but there's also a lot of methods to handle network traffic, like adding headers, validating requests.

hipster coder16:04:50

the authors say that if the language can run on the jvm, then it can run inside vert.x

hipster coder16:04:10

it uses jruby to run the ruby code, for example

dominicm16:04:19

Oh! So it is nashorn or something for js?

hipster coder16:04:15

that’s a very good question. I have no idea how it transpiles Javascript to the JVM

hipster coder16:04:09

this article uses npm to setup the JS stuff on vert x. But I am not sure if its actually using the node js runtime…

gklijs16:04:20

I guess, you can also use it to serve static files. But when they started the project, it did not support redirects. Which is why we have an additional layer of nginx. It would be much nicer/easier if we didn't had that in between.

hipster coder16:04:25

you couldn’t send a 301/302 redirect header to the browser?

gklijs17:04:02

not in the beginning they said..

hipster coder17:04:36

mvn clean install
worked

vemv19:04:47

seems rather uninformative. Is there a good rationale + arch page?

hipster coder21:04:47

I think the rationale is that you don’t need to worry about the common concurrency problems with deadlocks, race conditions etc… because you are using an actor style message bus… but vert.x can scale vertical or horizontal. And it doesn’t force you to use the async event loop for everything. It has a background worker pool for threading.

hipster coder21:04:16

It’s also modular. It’s not a monolithic frameworks that takes over your code.

gklijs04:04:26

Yes, but the bad thing is that's it's not scaling automatically. You really need to put in a lot of work to get high performance. Using kubernetes for scaling is probably both easier, and then it does auto scale.

hipster coder19:04:20

Does it require a lot of work to setup clustering with hazel cast? So you can scale across multiple servers?

gklijs20:04:49

Not sure as I don't have any experience with hazel cast. But the problem is you need to configure how many of each 'verticle' you want. Which takes time, and depends on the use. While these days I think it's much easier to use Kubernetes and auto-scale.

hipster coder21:04:57

Ahhh. So there’s not an easy built in auto scaling tool with Vert.x? One must use a 3rd party auto scaling tool? Although, I think almost any language would need an auto scaling tool? Right? Or wrong?

hipster coder18:04:34

Omg Chaim Kirby in the Python slack just told me to try using

git bisect
… it’s saving me hours if not days of work

👌 4
Daniel Hines20:04:44

This seems like the coolest thing in the world. Does anyone have any insight as to why it failed so hard? https://en.wikipedia.org/wiki/WinFS

Lennart Buit20:04:26

I think I read somewhere that it was cut from the ill-managed Vista

Lennart Buit20:04:51

yeah — I guess because Vista wasn’t coming along, adding a new FS was maybe too ambitious

Daniel Hines20:04:09

I guess timing is everything. But the idea itself… Seems like it would redeem the desktop as a platform for general purpose productivity (as opposed to the web apps like G-Suite that are currently winning).

Alex Miller (Clojure team)21:04:38

Didn't BeOS do this a zillion years ago?

john21:04:50

That way you can make desktop apps just like you make server apps!

john21:04:10

Isomorphic! 😉

Daniel Hines21:04:24

This is cool, thanks! Shows there’s really nothing new under the sun.

Alex Miller (Clojure team)21:04:59

computing was done in 1970 I think

Daniel Hines21:04:05

It really seems like it! I’m a wee youngin’, so all this is news to me. But Clojure and the community are constantly pointing backwards when I ask where the cutting edge is. I mean, it’s 2019 and ya’ll have got me programming in LISP on Emacs and watching lectures from when shouldpads were cool.

👍 4
john21:04:32

I suppose you'd want to be able to also view the db as a file system too, for backwards compatibility with other stuff?

Daniel Hines21:04:58

Yeah they seemed to exist in parallel.

Daniel Hines21:04:52

Far-flung spitballing, mostly for reaction:People do Datomic+S3 for blob storage. Could you do the same thing locally and implement a whole OS? All non-blob data in userspace (kernal/graphics aside) would be represented as EAVT tuples. You can look up blobs in the file system as you need to. OS ships with a default ontology/schema, but can be extended just like Datomic schemas. The OS API for normal user applications boils down to mostly two API’s: query and transact. Thoughts?

Lennart Buit21:04:43

isnt this basically the idea of a CoW filesystem

Lennart Buit21:04:54

if you write to it, you actually assert a “new file”

Lennart Buit21:04:32

only when there is no space left, a filesystem like Btrfs/ZFS will start clearing unreferenced files

Daniel Hines21:04:44

Well, I’m immutability is an essential ingredient, but I’m more talking about how data is represented, both in memory and on the disk.

Daniel Hines21:04:32

like, your email app and word processor, and all the rest have shared access to a datalog store.

Lennart Buit21:04:55

right, so your file system is no longer a ‘bunch of blobs’

Daniel Hines21:04:54

?! Maybe?! Time to start a reading list…

Alex Miller (Clojure team)21:04:01

has some of these ideas in it at least

Daniel Hines21:04:57

Windows has COM objects, but the problem is the same problem that the entire stack has - all the interfaces are ad-hoc, so you have no universal traction on them.

Daniel Hines21:04:08

You have to write new code for each and every interaction.

wekempf12:04:30

Actually, a huge part of what made COM so awful was that there was a huge amount of standard interfaces you were supposed to understand and implement on your components. These interfaces were all complex and hard to work with.

wekempf12:04:51

There's certainly more important reasons why us old timers shudder when COM is mentioned, but the idea that all COM coding involved ad-hoc interfaces (any more than any API has ad-hoc interfaces, BTW) totally misses the point.

Daniel Hines21:04:33

I suspect it would be possible to patch things up in a way that smoothed over remote connections, such that apps had no concern whether queries/transactions were resolved locally or against a remote DB somewhere.

wekempf12:04:58

Plenty of technology has attempted to make what you describe a reality and all have failed, for one important reason. The wire is not reliable or latency free. You can never treat it the same as local calls no matter what RPC mechanism you build around it. For this reason, it's better to not try and make remote calls seem to be the same as local calls.

Alex Miller (Clojure team)21:04:45

you are always concerned with that difference

Alex Miller (Clojure team)21:04:00

most of distributed computing is about the set of tradeoffs you can make in between those points

Daniel Hines21:04:33

I’ve done little distributed computing, so perhaps I can’t understand. However, network drives seem to work really well, such that garden-variety apps can remain ignorant of their remoteness. Wouldn’t the same thing be possible for more structured data?

hiredman22:04:54

there is a famous list of assumptions about distributed computing that have turned out to be false many times https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

Alex Miller (Clojure team)22:04:28

the history of computing is littered with the corpses of local/remote abstractions

Alex Miller (Clojure team)22:04:46

which is not to say they aren't useful

Alex Miller (Clojure team)22:04:29

but you should have a skeptical eye towards magical claims :)

hiredman22:04:34

rich has a talk, something about the system? maybe language of the system that talks a little bit about the trade offs you make when you make things network transparent (like erlang)

Alex Miller (Clojure team)22:04:44

yes, language of the system

Alex Miller (Clojure team)22:04:53

of which a key suggestion is to embrace queues as a tool for connections