Fork me on GitHub
#clojure
<
2022-12-14
>
leinfink08:12:29

Is there a pretty printer that visually distinguishes between lists and sequences? (I hope that's not a terribly confused question.)

pithyless08:12:00

portal and reveal do - which is fine if what you really need is a debugging tool, but they’re not pretty printers to stdout

pithyless08:12:55

actually, I think I may have jumped the gun - I’m on the phone and can’t check a repl - so feel free to ignore my response :)

pithyless08:12:30

I do remember reveal demo making the point of color differentiating two types that pprint to the same thing

vlaaad08:12:59

What do you mean by sequences?

leinfink08:12:10

like a lazy-seq if i am using a list and put it into some function the result may still look like a list in the REPL even though it has ben seq-ed

leinfink08:12:26

which i guess doesnt matter for lists but then i might not know intuitively what would happen if i had put in a vector instead, would it return the vector or a seq

leinfink08:12:37

thats was my thought-process :D

leinfink08:12:12

@U05476190 thx, i'll check it out

leinfink08:12:01

basically one of the more common mistakes i make so far in my learning journey is trying to use some data structure specific function on what turns out to have already been turned into a LazySeq, and I was looking for ways to help me realise when this happens

leinfink08:12:14

i guess i should just learn that it happens in most of clojure.core

reefersleep09:12:09

That was a hard lesson for me. I was very annoyed with it for a loooooong time when I started out learning Clojure. But yeah, expecting lazy seqs from Clojure is a good rule of thumb 🙂 And I’ve Stockholm Syndromed my way into kind of digging it? I certainly see benefits to lazyness.

leinfink09:12:49

oh, i love it! 😀

reefersleep09:12:49

In the same vein, I find it makes sense to kind of mimic what Clojure core does in most cases. And I find that that naturally happens when your functions are generally compositions of Clojure core (in my case, mostly threadings 😄 )

👍 1
jpmonettas11:12:09

maybe you can use something like this :

(defmethod print-method clojure.lang.PersistentList [l ^java.io.Writer w]
  (let [fmt-red (fn [s] (str "\033[31;1;1m" s "\033[0m"))]
    (#'clojure.core/print-sequential (fmt-red "(") #'clojure.core/pr-on " " (fmt-red ")") l w)))
which is as hacky as you can get XD

jpmonettas11:12:29

but then you will have lists in red

leinfink12:12:04

wow, cool, that's exactly what I had in mind

leinfink12:12:36

thanks, that worked @U0739PUFQ not red for me, but still noticeable

jpmonettas12:12:06

nice, I guess the coloring depends on your terminal interpretation of the escapes codes there

vlaaad12:12:43

Maybe I should add different colorization for persistent lists vs seqs

vlaaad12:12:54

In reveal I mean

skylize13:12:58

> i might not know intuitively what would happen if i had put in a vector instead Lists and seqs are both sequential access starting from the head. If you are trying to do vector-specific transformations, then you need to make sure you are starting with a proper vector.

=> [(type [:foo])
    (conj [:foo] :bar)
    (type (seq [:foo]))
    (conj (seq [:foo]) :bar)]

[clojure.lang.PersistentVector
 [:foo :bar]
 clojure.lang.PersistentVector$ChunkedSeq
 (:bar :foo)]

leinfink14:12:18

I know, that's what i meant. let's say I have (conj (foo '(1 2 3)) 1) ; => (1 2 3 4) , then I decide, ah, no, I want a vector so i can conj at the end, (conj (foo [1 2 3] 5) ; => (5 2 3 4) ah, damn, apparently it turned into a seq at some point in foo. but just from looking at the repl, i couldn't know that when i was still operating on the list

Joshua Suskalo14:12:06

So if the problem is confusion between lazy sequences and data-structure-specific operations, then lists and sequences don't need to print differently. lists and sequences have the same set of data structure ops work on them in the same ways. The bigger problem is just remembering which functions return lazy sequences and which return data structures, and the general rule for that is that any function that takes the collection as the last argument and returns a collection is returning a lazy sequence (except for ones replicating those apis for vectors, see mapv, filterv). If a data structure function takes the collection as the first argument, it will preserve the type of its argument.

2
Joshua Suskalo14:12:23

This pattern doesn't always carry to library functions, but it's true in general for core.

leinfink14:12:38

that's a helpful tip, thank you

phill22:12:13

Emerick, Carper & Grand's book helped me distinguish the "sequence library" part of clojure.core - a quantum leap in the learning experience - sort of like when you are staring at the page of dots and suddenly you see the airplane leaping out of the page at you. But nowadays, if you don't need laziness you can use transducers and explicitly choose the output container, which is another way of not being surprised by it!

emilaasa15:12:49

I'm going to store some state on disk between runs of a long running job. I'll be storing the job progress and possible error states of a few million records that I'm updating. I will need to access this state concurrently, and it will contain an url to a slow service that I will download some data from, transform it and send it away, then update the job state to "SUCCESS" or something. When I've done similar things in Python, I've reached for sqlite because I'm pretty comfortable with that, and the transactions are nice. I thought it could be fun to do it with Clojure somehow. Any ideas on how to construct such a thing?

Rupert (All Street)17:12:46

Reading and writing from disk works well form Clojure. A few formats to consider: • Binary: Nippy / MessagePack • Plain text: EDN / Transit / JSON • DB: SQLite, Lucene A few thoughts: • File system can handle handle lots of files and folders - so feel free to create lots of files and then delete them later as part of a clean up job. • Rename/move operations are guaranteed atomic so use this to your advantage (e.g. rename/move to signal a file is ready). • The disk cache will mean that recently written files are very fast to read. • You can list files in multiple folders with file-seq or you can be alerted to changes with a library like https://github.com/wkf/hawk (but you may need to tweak linux settings to scale this).

Huahai01:12:39

try datalevin. It’s more convenient than sqlite, just a clojure library.

emilaasa06:12:33

Thanks for your replies! Datalevin looks really interesting.

Huahai06:12:12

Happy to answer any questions. There’s also a #C01RD3AF336 channel here.

elliot15:12:36

Is there anything newer/better than stuartsierra/component these days for managing stateful system lifestyle or is this still the sort of common pattern?

Joshua Suskalo15:12:24

Component complected API with dependency/state management. Integrant is very good and only deals with dependency/state management, it doesn't touch API.

Joshua Suskalo15:12:35

Well, technically Component doesn't inherently complect those, but the way that it's most commonly used does.

elliot16:12:02

Hrm very cool thank you!

jjttjj16:12:19

There's a new one every week 🙂 Jury is still out on which is "best". I think someone attempted to compile a comprehensive list of them somewhat recently I'll try to find it

elliot16:12:57

amazing thank you!!

jjttjj16:12:26

I think it looks like something that is highly developer-preference-specific, and a thing where you can spend a day to make a little library for your exact preferences after repeating them in a few projects, so a lot of people have done that.

elliot16:12:33

yeah fascinating. That totally makes sense in practice. It is kind of interesting how this seems to keep slightly defying standardization in the language/ecosystem. So many things are beautiful/clear/standard in the language (e.g. functions on maps, state through atoms etc.) but these questions of whether components are implicitly stateful functions (closed over their dependencies) vs. explicitly functions of state, and how to represent dependencies between them, seem a little unresolved yet

elliot16:12:08

Looking forward to try to check through what people have come up with in recent years. Thank y’all again!

Joshua Suskalo16:12:44

I suspect that this is because a lot of developers see no need for one of these, and the people who do all have very different ideas about what it should look like, and so you don't see standardization because there's no critical mass of developers that don't care about which solution to use for DI, just that they use a solution for DI.

👍 3
Joshua Suskalo16:12:44

and so because there's no critical mass of people that don't care about what the api looks like, you don't get almost anyone looking for "the popular one", and so you don't get any significant knock-on effect except for component coming first.

👍 1
robert-stuttaford16:12:50

we use and love integrant

🎉 2
robert-stuttaford16:12:09

it pairs up brilliantly well with aero

robert-stuttaford16:12:58

(10 year old 150kloc Clojure/Cljs/Datomic codebase)

jjttjj16:12:57

This was also a pretty good recent related article: https://mccue.dev/pages/12-7-22-clojure-web-primer

🙏 1
seancorfield17:12:48

And to counter Robert's comments, we use and love Component, and use it with a 10 year old 133kloc Clojure codebase. So both libraries will satisfy any future needs you may have. I prefer Component over Integrant because it is much simpler and smaller (just two lifecycle function). But I'm also a fan of metadata-based protocol extension, so we use Component on things that aren't records and we also often make our Components "invocable" (by extending IFn) to yield their natural underlying state. next.jdbc takes a similar approach https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/connection.clj#L304-L313 by using metadata on a function -- so the function is the Component (and can be invoked to get the underlying datasource) and you can call stop on the function via the protocol metadata.

lukasz18:12:57

+1 on Component too, I'm still not sure why people have problems with it, especially after extend via meta was added. I kinda wish we stopped reinventing the wheel

robert-stuttaford18:12:11

what i hope a reader takes away from this is that the approach is sound, and the implementation is to taste 🙂

1
☝️ 1
Ben Sless18:12:33

On the topic of component, this library isn't exactly officially released yet but it's my attempt to find an escape hatch from the law of preservation of bummer https://github.com/bsless/companion

nice 2
lukasz18:12:22

pretty cool, I wrote the IFn extension, but adding support for deref looks like a great idea, gonna steal it 😉

Ben Sless19:12:24

Just be mindful of the license 🙃

seancorfield19:12:54

> I kinda wish we stopped reinventing the wheel I think this is because all the solutions to this problem are fairly small pieces of code that just offer different affordances: some people like Mount because it follows the pattern of namespaces for state and automates the start/stop dependency order so it's intuitive (and, dare I say, "easy"); some people like Component because it's small and simple (and perhaps feels like encapsulation of state -- which makes it easy for tests to spin up multiple instances of services as needed); some people like Integrant because it's very data-focused; some people like other approaches. You can solve this problem entirely manually for smallish codebases, so it's not like any of these libraries are necessary for a lot of people.

Ben Sless19:12:11

One thing I leant into with companion is making a ton of tiny pure data components, it actually made the application more configurable in conjunction with aero

seancorfield19:12:36

Makes a note to go check out Aero 🙂

Ben Sless19:12:55

Aero is awesome

5
lukasz19:12:54

I'm sketching out something that marries Component and Aero, but it's too early to show anything

Ben Sless20:12:25

Don't they already work?

Ben Sless20:12:47

just merge-with merge

markbastian20:12:59

I've been using Integrant for years and swear by it. If you're trying to make the distinction, I'd say the primary difference between Component and Integrant are Component is record-based with protocol-based dispatch and Integrant is data-based with multimethod-based dispatch. I prefer the data-oriented approach, but, as you can see here, both systems are pretty solid. I don't recommend mount.

seancorfield20:12:19

> Component is record-based That's not really accurate. You can use any data/objects with Component, with two caveats: if something has dependencies, it must be associative by keys; if something has a lifecycle, it must be able to carry metdata.

seancorfield20:12:42

(to be clear, it used to be that Component was record-based)

👍 1
1
markbastian20:12:15

Thanks for the clarification.

seancorfield20:12:46

I talked to Sierra about extending Component to support dependencies via metadata so you didn't have to use associative data for that, but they said "that would be a different library" which is fair enough.

pablore21:12:33

Hello everyone, long time no see! I came here looking for a library that can create clojure datastructures from the table schemas of a sql connection. Something like:

Table User
| name    | email      | id  |
+---------+------------+-----+
| varchar | varchar    | int |
Into:
{:name :str
 :email :str 
 :id :int}
I know that I can dwell into jdbc MetaData, but I wanted to know if there’s already a tool that does it.

rolt21:12:31

https://github.com/donut-party/dbxray is that what you're looking for ?

🙌 1
pablore21:12:28

This is exactly what I was looking for! Thanks!