This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-04-20
Channels
- # announcements (3)
- # babashka (7)
- # beginners (36)
- # calva (71)
- # cider (25)
- # clj-commons (5)
- # cljdoc (19)
- # cljs-dev (5)
- # clojure (223)
- # clojure-austin (2)
- # clojure-bay-area (1)
- # clojure-europe (31)
- # clojure-france (6)
- # clojure-nl (2)
- # clojure-norway (19)
- # clojure-spec (13)
- # clojure-uk (7)
- # clojurescript (127)
- # core-logic (2)
- # cursive (21)
- # datalevin (53)
- # datomic (9)
- # emacs (37)
- # events (1)
- # graphql (8)
- # jobs (12)
- # lsp (8)
- # off-topic (92)
- # pathom (49)
- # pedestal (1)
- # polylith (3)
- # re-frame (25)
- # releases (2)
- # sci (11)
- # shadow-cljs (13)
- # vim (10)
I can't think of the name of this. It's a command line tool that says it's like git for your files. You can checkout one subdirectory of the remote and get the files somewhere locally. You can then check the files in at a later time. I think it has "git" in the name. In reality, it doesn't have any connection to git. I'm not able to google for it... Thanks!
TIL... There's also git lfs (which seems to be more widely supported). Apparently both solve the same problem.
Ha, I think I originally learned about git-annex, from this guy named Richie: https://www.youtube.com/watch?v=1GKjKCUwwtY&t=619s Hopefully that's not you, having forgotten the tool 😄
I believe git-annex is more feature-full and flexible than git lfs, but if lfs is enough for your use case then I'd go with that as it is likely to be perceived as an industry standard way. annex is very robust, thanks to it being written in Haskell I imagine (and props to the developer JoeyH). So it might simply be "done" and not need any update, but I still lament that it is feeling a little "old" and not really active. Around a few things I haven't managed to get quite the workflow I wished for: I believe there are some issues around add-url / get in combination with the outdated quvi tool, which doesn't quite play well with modern youtube. HTH.
Someone posted a learn datomic website about a week back and for the life of me i can't find it. It had a really cool front page. 🙂
max datom? https://max-datom.com/
yea! thanks!
This intro story is intense. I'm really hooked, and slightly worried i'm actually in the Matrix.
The greatest complement i have ever received: > You seem to excel at the unskilled task of pushing a pre-populated button.
I got a polo shirt, i'm really part of the team now!
I'm just getting around to trying this today and it's kicking my ass. I've never actually used Datomic or the like, I just skimmed through the docs (and the learndatalogtoday tutorial) and told myself "yeah, that makes sense". What a wake up call!
let me know if you have any questions @U8LB00QMD i found some parts hard to grasp because i don't think the expected output was clearly defined. Small things like "in what order" mattered. I suspect it might have gotten improved sense i did it though 🙂
> If you have deployed Java 15, Java 16, Java 17, or Java 18 in production then you should stop what you are doing and immediately update to install the fixes in the April 2022 Critical Patch Update. https://neilmadden.blog/2022/04/19/psychic-signatures-in-java/
I am also a little confused about this. Maybe it’s too abstract, or lacking a fancy name like “log4shell”? We have taken (some non-critical) applications offline temporarily while we wait for the Temurin Patch
postgres+++++
no they're + like in C+++++ 🤪
Ecto is amazing. It's the closest thing to an ORM that I enjoy working with on the daily
that does look neat
I suspect my tastes with this will differ a lot from most clojurians, though
Ecto is indeed interesting- but IIRC it needs to take over the schema, right? Or does it introspect a database?
Gungnir is somewhat similar, I think. https://github.com/kwrooijen/gungnir a bit more complicated though. I don't know where this project ended up.
It doesn't need to take over the schema. You retain full control of your database. You don't have to buy into the whole library. You could use Ecto.Query only as a query builder, but most people also use Ecto.Schema, where you can map structs to your database tables, encode relationships, etc.
Currently we are only using honeysql to read from Postgres so I haven't had the need to deal with anything like eg Ecto changesets. I don't feel like I'm missing a query builder
the Ecto query builder is pretty much equivalent to honeysql anyways, you're not missing out.
Ecto Changesets are nice in the way that you can describe different update operations on a schema and enforce some validations in some cases but not others. They also make change tracking very easy. If you map a set of attributes to a struct, you will know exactly what changed without having to figure it out manually. Relationships and associations are a bit of a leaky abstraction, but are very useful if you can work with them in the intended way. You can manage a pretty deep graph of models quite trivially.
Isn't toucan similar to ecto?
Also I've got some experience with ecto but nothing can take the Django orm 's place in my heart
I was a fan of Ecto at first, but over time started to dislike it. It works great for structuring updates in simple use-cases, but becomes a nightmare when your data starts to have sub-arrays, sub-objects, etc. The whole thing of using custom functions to get values, making dynamic access harder than it needs to be for "safety", etc., - it just creates a lot more problems than it solves imo. That said, in Clojure you could probably fix all that easily. I think the important thing though is to keep stuff separate - don't mix existing data, changes, and validation errors in the same sub-paths. Put those as top level keys instead, it will make it much easier to deal with.
@U08JKUHA9 I found that most of the problems you mention come down to bad schema design. You should be very rarely relying on assocs and embeds. It's very easy to fall into the ORM trap of trying to schematize every possible relationship in your database. We don't do that anymore. Our models are much more decoupled and we're far better off for it.
Sometimes it is within the same table though - Json. And I'm sure people have different opinions on schema design, but I'm just saying that Ecto design creates problems without any big upsides.
I agree that ecto doesn't support complex queries very nice. You have to use fragments all the time.
And your suggestion sounds more like a workaround than disputing that the design is problematic.
Try to compare this with the Django orm. There are years since i had to use raw queries
It more or less support everything.
Look at this work of art for example https://github.com/spapas/etsd/blob/master/etsd/msgs/views/__init__.py#L91
Or this https://github.com/spapas/etsd/blob/3085b6fba5fc26f11132b1eafaf3e5eab3155996/etsd/core/views.py#L408
By the way, about BeeRecord (the one I'm slowly developing) - it leverages HoneySQL to convert to SQL, and Pathom3 to make the queries. You can use both by only avoid boilerplate (it tries to join things to you, and spits a HoneySQL code that you can assoc and dissoc as much as you want) or you can go full-blows and make the queries (it'll try to avoid N+1 by using Pathom's Batch Resolvers). What I want to do in the future is to expose more things. For example, suppose you have the "database mapping" done, and you want to expose that some code. Instead of using as an ORM, you could just return the Pathom resolvers and add your own at the process - in that sense, BeeRecord's kinda become "one step on the Graph" and you could, theorically, join REST + SQL with ease; or you could integrate with Lacinia and convert everything to GraphQL too. Another thing that I want to add is interceptors - for example, instead of "make the query, query the DB, return the results" you could "make the HoneySQL structure, then add an interceptor to change something, now make the query but with these parameters and with this library, then convert the result to something else". This may be easy to do - after all, is all EDN structures
"work of art" is very debatable here. I'm not familiar with Django nor Python for that matter, but to me it just looks like a jumbled mess. I'd much rather drop down to raw sql for that kind of query.
prime example of too much abstraction IMO, it distracts from the underlying thing by making it more complex. but hey, that's my 2 cents and mine only. no judgment at all here.
What I mean is: BeeRecord is not an "ORM", and it does not try to hide SQL to you. It indeed have a "default options" but you can customize almost everything, and that's the feel I want for this mapper 🙂
Marco for me the work of art is the fact that you can express a very complex query using the orm without resorting to raw queries
You may don't like it but using the orm instead of raw queries is invaluable in lots of situations
That's not a compelling argument in favor of ORMs for me but I see where you're coming from! It does indeed look very powerful.
I don't think there is a great argument if the query is static. It is once some parts of the query need to be dynamic then having a query builder can be easier than concatenating strings. Another case would be re-using query fragments.
I'm doing very complex queries with honeysql (joins, ctes, calling functions, aggregations and other more or less exotic sql features) and never have I thought “this would be easier with a query builder”. And I bet it'd be very discoverable by anyone who knows SQL.
And everything being Clojure data, we have a ton of functions that operate on those queries e.g. to add permissions, add/remove select columns etc.
Well it's operating directly at the SQL abstraction layer, it doesn't add an abstraction layer on top.
What's your opinion on toucan?
@U03BMTULBSL toucan looks way too opinionated and constricting for my tastes. Single implicit db connection is a major red flag. Some nice ideas about hydration and some lifecycle functions (eg converting string to keyword and back). Built on top of semi-obsolete libraries is also a bad sign… Both Clojure.java.jdbc and honeysql 1.0 have successors.
there's a lot to like there, especially for very simple apps
I kind of feel like sql just isn't easy enough to compose and anything you come up with is going to involve some annoying trade-offs. the more abstraction and indirection you add to make sql composable the further you are from the generated sql and the less transparent your code will be and the harder it will be to land at the right sql. if there's no abstraction or indirection then you end up writing every query by hand entirely, even very simple things. somewhere with in between the extremes is going to make different parties unhappy in different situations
add to that how opinionated every clojure developer is and you end up with a proliferation of libraries that make everyone unhappy in one way or another 🙃
it's enough to make you go off and build your own database! not that we clojurians would ever do that
i'm being cynical to be funny, really. I'm glad there's a little something for everyone and that clojurians often try to solve this problem at the root 💜
The thing is that you can leverage Honey SQL to get you 80% there in 100 lines of code that you can throw together in an afternoon. So the impetus to look for, or even create a library is much smaller than other ecosystems.
Thank you very much for the answers!
I guess I'll start with honey sql ; I've heard good things from many sources
Is it just me that find insane how some kind of software is developed? For example, frontends with Javascript most of the time use lots of dev-packages
that are removed on production builds; this translates to "I have a build config in development and one in production". But then, it needs to run tests, so there's a third build that is only for tests, and uses Jest to mock the entire world so you don't depend on a real browser, for example... and now you have your tests running on a mocked environment, on Node.JS (AKA a different Javascript, and essentially, a language than the browser it's supposed to run), with mocked server interactions so you don't catch bugs if the server changes the payload, even by a little bit, and most of the time these tests don't even bother to check if the components/divs/web elements change if someone clicks on a button...
This seems to be an architecture problem more than a language problem. Relying on mocks during testing might mean there is too much IoC for no reason other than stuffing side-effecting things into domain logic.
In Javascript-land, it's the norm - with probably the exception of node.js projects, almost everything is tested in a completely different configuration...
I tend to test against a real system if possible and use as few mocks as possible
I sometimes like to do boxes with mocked UI separately (for instance with typescript interface) and then add logic into it later. Button clicks and something like that. Sometimes.
Yeah, especially if you're using tools like Terraform/Fargate/Kubernetes to declaratively provision infrastructure and deploy code on it, the effort required to mock things could as easily be spent on developing tests that target an actually running system that replicates production. Bonus: your test suite now has a concrete dollar cost, incentivizing you to focus on eliminating tests that don't add value 😉
Hi, guys do we have performance channel? Did anybody experience locking with atom - looks like shared atom (as cache) is a bottleneck in our server benchmarks
Atoms are lock free,, so I doubt your experiencing that. Maybe you've got many things trying to update at the same time? Then you'd have a lot of failed swaps doing a lot of retries.
Concurent hash map would probably be faster in that case. But the semantics are different. It doesn't let you atomically read -> update -> write. It only lets you write atomically. The data you read could be stale basically. It's like always using reset! with an atom basically.

But it should be faster as it locks on segments. So writing to parts of the keys will lock but writing to other segments of them concurrently will be allowed.
The #clojure channel is for Clojure users. The #clojure-dev channel is for development of Clojure. It's perfectly normal to ask #clojure performance problems in #clojure.
People ask for even waaay simpler things there. From how to add numbers to your question.