Fork me on GitHub
#clojure
<
2019-01-06
>
Audrius10:01:30

I have working code where I get hash code of data structures with (hash data-structure) but it works in small amount of them. When I have few millions of data structures it will start giving me duplicate hashes (clash). How to get something more unique when structure of data structure is unknown?

reitzensteinm11:01:26

@masta hash returns a 32 bit int, so the birthday problem means you'll start to get conflicts regardless

reitzensteinm11:01:06

You might want to look in to something like sha or md5

reitzensteinm11:01:40

Or design your data structures to work even with hash conflicts, which is what clj itself does

5
borkdude11:01:45

how do I get pprint to print this with the leading quote?

(def x '[foo bar])
(pprint x)
=> [foo bar]

5
jeroenvandijk09:01:05

Like this?

(clojure.pprint/pprint ''[foo bar])
'[foo bar]

borkdude09:01:41

can you do that with the x instead of the double quoted vector?

jeroenvandijk09:01:25

Maybe not. Is it for edn data or for something else?

borkdude09:01:47

code generation, but I already have a solution, you’ll see it if you read on 🙂

👍 5
jeroenvandijk09:01:13

Or indeed

user=> (defmacro quoted [x] `(quote (quote ~x)))
#'user/quoted
user=> (def x (quoted [foo bar]))
#'user/x
user=> (clojure.pprint/pprint (quoted [foo bar]))
'[foo bar]

jeroenvandijk09:01:32

cool, i'm curious

borkdude09:01:03

I did it like this: (list 'quote all-syms-clj)

borkdude11:01:53

(pprint (list 'quote x)) seems to work

mfikes15:01:47

@borkdude Not sure if it is an idiom, but I often see the sequence

`'~
used in macros. It looks a bit like line noise, but perhaps it is an idiom. So for that example
(pprint `'~x)

borkdude15:01:41

cool. I have used it in macros yeah

nicoschneider18:01:40

Hey everyone, I have a problem with leiningen, reloaded.repl, Cursive and/or CLJC. I have defined some protocols in .cljc files which are used from clj/s. In an open REPL from Cursive, most of the time when using reloaded.repl/reset(-all), it doesn't pick up my changes at all. If I do a "manual" clojure.tools.namespace/refresh(-all), my changes are picked up but all the protocol implementations won't satisfy their respective (newly modified) protocols anymore, so I get exceptions from somewhere near clojure.core/-cache-protocol-fn. I need to fully restart the REPL for everything to be picked up again. Is there a way to remedy this, or workarounds? Has anybody encountered this? (my google searches haven't turned up much)

nicoschneider18:01:05

(My workflow is roughly equivalent of what using chestnut involves)

lilactown18:01:48

this sounds relevant: https://lambdaisland.com/blog/2018-02-09-reloading-woes > when you use tools.namespace to reload your code, it effectively creates completely new and separate versions of functions, records, multimethods, vars, even namespace objects, and any state that is left around from before the reload might still be referencing the old ones

lilactown18:01:08

I personally don’t use tools.namespace, but I know that re-evaluating things like multimethods is tricky

lilactown18:01:18

I wonder if records/types that implement protocols is the same

lilactown18:01:09

have you had any success simply re-evaluating the namespace you’re trying to refresh?

nicoschneider18:01:37

Well, sort of. It sends me on a wild goose chase through my namespaces, totally arbitrary and non-deterministic

nicoschneider18:01:25

Sooner or later it sometimes works, but not so often that restarting the whole thing isn't much easier and faster

nicoschneider18:01:42

Multimethods work fine, by the way

nicoschneider18:01:22

I have lots of defrecords implementing various protocols, and those very often "loose" their implementations in this way

lilactown18:01:49

well, specifically, I always run into an issue where you can’t re-evaluate the (defmulti <dispatch>) part. re-evaluating the cases works fine 🙂

nicoschneider18:01:50

which is a bummer since I quite like the architecture so far, but without proper refreshes in the REPL it's very frustrating

nicoschneider18:01:13

ah you're right, I know that one too ^^

nicoschneider18:01:00

Maybe it's reloaded.repl's alter-var-root! that does something funky?

lilactown18:01:06

so, question: if you create a new instance of the new defrecord impl, do you see the correct behavior?

lilactown18:01:08

after a reload

lilactown18:01:04

(I’m also assuming that you’re interacting with this at a REPL to some extent)

nicoschneider18:01:48

iirc yes, if I switch to the respective namespace and manually instantiate the record, it satisfies the protocol

nicoschneider18:01:18

but let me check real quick

nicoschneider18:01:56

(most of the records are instantiated via an integrant like multimethod mechanism)

lilactown18:01:26

so it sounds like what’s happening is that your defrecord is being picked up correctly, but your application is still referring to instances of the old implementations

lilactown18:01:53

I haven’t use integrant, I typically use mount because I’m lazy. but I assume there is a way to stop/start your system with new instances of your records, which would solve the problem you’re encountering I think?

nicoschneider18:01:59

i'm not using integrant, but the core of the IoC mechanism is a multimethod that's basically integrant's init-key

nicoschneider18:01:03

i'll look into the REPL utilities to see if there's something that keeps around stuff

nicoschneider19:01:56

I'd be surprised though since afaik there's one aliases and stuff. reloaded.repl uses alter-var-root! and component/suspendable, so in a reset, it'll call Lifecycle/stop. All the records in question are re-instantiated on start and discarded on stop. But there could be a bug there as well I suppose.

nicoschneider19:01:05

there's only aliases*

lilactown19:01:25

hm. oh, so you’re using component?

nicoschneider19:01:38

yes, but only "behind the scenes"

nicoschneider19:01:25

and really only for repl convience. all components wrap a subsystem which consists of the actual parts of the application

nicoschneider19:01:59

the wrapper components discard the subsystem on stop and create a fresh one through the IoC mechanism on start

nicoschneider19:01:18

which is why this protocol issue puzzles me

nicoschneider19:01:13

reloaded.repl's reset does stop, refresh, start, so in theory the subsystems should be created after the tools.namespace refresh

nicoschneider19:01:28

and with the new implementations

nicoschneider19:01:02

but maybe that's the crux, a combination of cljc processing and protocol/multimethod reloading

nicoschneider19:01:18

or protocol reloading and multimethod instantiation, rather

nicoschneider19:01:42

the reason I suspect something very messy is that very rarely everything breaks completely, even after a REPL restart, and I have to lein clean before any of it will work again 😅

lilactown19:01:05

hm yeah this all sounds very complex

lilactown19:01:58

FWIW I don’t use reloaded.repl at all. I use mount and re-evaluate the var or namespace I’m in as I develop. Occasionally, if necessary, I call (mount/stop) (mount/start)

lilactown19:01:45

but I also have never used records and rarely use protocols 😛 so take my advice with a grain of salt!

nicoschneider19:01:01

we considered mount but decided on component+reloaded, since they seemed to be made for record/protocol stuff

nicoschneider19:01:38

and it works quite beautifully, too, except the actual reloading 😄

nicoschneider19:01:42

Well, I'll try reproducing this in a minimal test project i think. Maybe it really is something with the code. Otherwise maybe I can try using an atom for REPL development and completely scrap everything on restart, to rule out reloaded

👍 5
kwladyka19:01:14

Who can share experience about https://github.com/onyx-platform/onyx ? Do you feel it is right solution for PDF processing when fast response for query by API is needed?

WhoNeedszZz19:01:42

I remember seeing a comparison table of Clojure web libraries/frameworks. I can't seem to find it now. Anyone have the link?

WhoNeedszZz19:01:14

It showed which ones use macros and which ones are data driven

WhoNeedszZz19:01:46

Yes, that was it, thanks. Forgot it was just comparing routing specifically though.

WhoNeedszZz19:01:40

I'm contemplating bidi + yada, but not quite sure the practical differences between the libraries

kwladyka19:01:47

From what I see yada use bidi under the hood

kwladyka19:01:40

I don’t know this library but it looks like it is bidi + ring glue

WhoNeedszZz19:01:31

That could be. They are both from JUXT and is stated as sister libraries

WhoNeedszZz19:01:01

It is an alternative to ring though

WhoNeedszZz19:01:07

"A yada handler is an instance of the yada.handler/Handler record. Since this record satisfies clojure.lang.IFn, yada handlers behave just like normal Ring handlers and can be used wherever you might use a Ring handler. "

WhoNeedszZz19:01:06

Yeah, looking at that too

lilactown19:01:09

yeah. I think you would use something like: ring (http abstraction) + jetty (web server) + bidi (routing) + yada (resources)

WhoNeedszZz19:01:29

No, yada provides the http server and is an alternative to ring

WhoNeedszZz19:01:34

It uses Aleph

lilactown19:01:47

:thinking_face: my mistake. must have misread the docs

dominicm20:01:24

@whoneedszzz if you have any questions about bidi/yada, there is a subset of us specifically in #juxt

👍 5
dominicm21:01:55

Fwiw, it isn't outside the realm of possibilities for yada to work with other web servers, but yada is currently coupled to the async extensions that Aleph adds. We haven't had a need to extend yada to other web servers, so it hasn't happened, and in the mean time, we have added conveniences for starting yada.

orestis21:01:36

I can also recommend pedestal. Different aims than yada, still a good candidate.

WhoNeedszZz21:01:35

Thanks for the recommendations

john23:01:58

I try to avoid the life cycle (mount/component) stuff unless absolutely necessary

WhoNeedszZz23:01:42

Can you elaborate?

john23:01:56

I guess certain extremely complex webservers can benefit from it. But it seems like lots of templates include the structure just in case you need it. Which usually isn't a huge deal, since it doesn't usually get too much in the way even when you don't need it. But then it's just one extra piece of unnecessary complexity.

john23:01:11

And in my experience, it's quite easy for projects to evolve and reloading to become broken with life cycles, unless teams maintain good discipline

phill23:01:57

"Component" itself is an outline of the necessary discipline, no?

john23:01:20

It is. And I guess I'm glad the abstraction has some standard. I just think it's over used

phill00:01:43

definitely an enterprise use case

phill00:01:55

as opposed to small team

john00:01:20

Yeah, large enterprise services

WhoNeedszZz00:01:03

I guess I'm not exactly sure what your statement was responding to.

john01:01:53

My statement?

john01:01:19

I was just airing a general complaint, sorry. Your use case may be entirely appropriate.

WhoNeedszZz01:01:53

I'm just not familiar with mount or component and didn't see it mentioned above it so I was just confused by that. If I knew them I might agree with your complaint.

WhoNeedszZz01:01:13

From a cursory glance of them, it sounds to me like it's another example of OOP mentality trying to claw back in to our functional paradise. 🙂

john01:01:40

I guess I was reading some backlog in another channel... And I accidentally responded in this channel... Lol

WhoNeedszZz01:01:02

No worries. It ended up having a relation indirectly after all

john01:01:03

Now I can't find it 😁