https://stackoverflow.com/questions/58405558/how-do-i-load-data-structures-containing-atoms-stored-in-a-file-in-clojure

Guys... Can anyone please please help me with this?

I've been at it for several days...seems pretty trivial but still has continued to successfully and repeatedly want to smash my keyboard...the result being one of my command keys has popped out of the keyboard

"Guys" is kind of rude. There are 16,000 people here and a lot of them are not guys. Please use "Folks" or "Everyone" or "All" -- something inclusive.

(I'm one of the Admins here, BTW)

I am so sorry..I absolutely agree with you...that was insensitive of me..

Looking over your question, I think the simple answer is "You can't write atoms out like that to a file and read them back in".

You'd need to arrange for your neural map to write out plain data -- with the atoms dereferenced

What if I serialize and deserialize it?

Like in java

Hmm, I'm not sure how an atom serializes...

My approach would definitely be to transform the data structure to a plain Clojure data structure and use that for the persistent file format -- unless you have cyclic data? But I would avoid records and atoms etc for the data structure if possible.

I see...ok thanks...

I'll try some things out and see

If your data isn't cyclic it should be fairly straightforward. Otherwise you might need to synthesize some "labels" for nodes so you can write references out instead of cycles 🙂

I think fundamentally that's because an atom is a Place and not a Value, and Clojure doesn't allow you to simply serialize something that's a mutable reference to something else?

Yeah, serializing "variables", whether Var, atom, ref, agent, is not a default built-in operation. clojure.datafy has a way to render an atom as "pure data" but with complex metadata:

seanc@DESKTOP-QU2UJ1N:~/clojure$ clj
Clojure 1.10.1
user=> (require '[clojure.datafy :as d])
nil
user=> (def a (atom 42))
#'user/a
user=> (d/datafy a)
[42]
user=> (meta *1)
#:clojure.datafy{:obj #object[clojure.lang.Atom 0x5328a9c1 {:status :ready, :val 42}], :class clojure.lang.Atom}
user=>      

(so it still isn't suitable for serialization to a file directly but...)

@seancorfield solved it using handler functions...it is possible to save and load atoms from files directly

Posted it on stack overflow

Cool! Off to read that now. Thank you for posting back with a solution!

Instead of def (which always creates a top-level binding) inside defn, you probably want let to create local bindings.

Your solution will fail if anything other than an atom expands to #object within your data -- just so you're aware that solution is very specific to your current data model.

i keep getting this error

Error updating class definitions:
Execution error (NullPointerException) at java.util.concurrent.ConcurrentHashMap/get (ConcurrentHashMap.java:936).

evaluate *e to see full stacktrace?

is there a channel for manifold library?

or i just go right here?

don't know if it will help, but there is an aleph channel

there is. but it's almost dead.

perhaps it's worth trying here then :)

i expected to see this code to take about 1.5 seconds to complete, but it takes 3.5 seconds. obviously, i don't yet understand how to do concurrent programming in clojure properly.

(let [pool  (e/fixed-thread-executor 5)
		  xform (fn [xf]
					(fn
						([] (xf))
						([result] (xf result))
						([result value]
						 (xf result (d/future-with pool
										(timbre/info "Got " value)
										(Thread/sleep 500)
										(inc value))))))
		  ch    (ms/stream 5 xform)]
		@(d/future-with pool (ms/connect (ms/->source [1 2 3 4 5 6 7]) ch))
		(time (vec (ms/stream->seq (ms/realize-each ch)))))

Hi, I'm using sente and I'd like to expose an API through a /foo endpoint, but I have the "anti-forgery" middleware which makes my requests end with a Invalid anti-forgery token , is there a way to request the CSRF token from outside ?

And how can I use it with curl for example? (Because worst case I can dump to csrf token to a file when starting it)

https://github.com/edbond/CSRF answers the question

any best practices considering javadoc generated documentation for java sources and how to package that along your Clojure library?

Maven has a convention for this

you upload it with same coordinates, but classifier of "javadoc" (as a jar file)

what I would rather want is generate these docs so they become visible on github

somehow.

well you can use github-pages for that

right, was thinking about that

either on a gh-pages branch or in a gh-pages directory now

aah, thanks, I'll look into that

that's how all the clojure core docs are posted

https://clojure.github.io/clojure/ - is just a gh-pages branch in the clojure repo

https://github.com/clojure/clojure/tree/gh-pages

(this pre-dates the directory option)

cool, something works: https://borkdude.github.io/sci/javadoc/index.html

I actually wanted to suppress the _val method documentation but it seems javadoc doesn't have something for that

Now I'll have to figure out how to do a javadoc for a class generated with :gen-class 😛

Maybe I'll just write a wrapper class and be done with it

Is there a way to catch all exceptions. For instance, the following try/catch block doesn’t catch the exception:

(try (1/0)
       (catch Throwable e
         42))

3. Unhandled clojure.lang.ExceptionInfo
   (No message)
   #:clojure.error{:phase :read-source}
                  main.clj:  410  clojure.main/repl/read-eval-print/fn
                  main.clj:  409  clojure.main/repl/read-eval-print
                  main.clj:  435  clojure.main/repl/fn
                  main.clj:  435  clojure.main/repl
                  main.clj:  345  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj:  660  clojure.core/apply
                regrow.clj:   18  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   79  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   55  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  142  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  171  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  170  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  745  java.lang.Thread/run

2. Caused by clojure.lang.LispReader$ReaderException
   java.lang.ArithmeticException: Divide by zero
   #:clojure.error{:line 33, :column 12}
           LispReader.java:  314  clojure.lang.LispReader/read
           LispReader.java:  216  clojure.lang.LispReader/read
           LispReader.java:  205  clojure.lang.LispReader/read
                  core.clj: 3768  clojure.core/read
                  core.clj: 3741  clojure.core/read
    interruptible_eval.clj:   99  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  410  clojure.main/repl/read-eval-print/fn
                  main.clj:  409  clojure.main/repl/read-eval-print
                  main.clj:  435  clojure.main/repl/fn
                  main.clj:  435  clojure.main/repl
                  main.clj:  345  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj:  660  clojure.core/apply
                regrow.clj:   18  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   79  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   55  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  142  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  171  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  170  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  745  java.lang.Thread/run

1. Caused by java.lang.ArithmeticException
   Divide by zero

              Numbers.java:  188  clojure.lang.Numbers/divide
           LispReader.java:  512  clojure.lang.LispReader/matchNumber
           LispReader.java:  350  clojure.lang.LispReader/readNumber
  ...

1/0 is a read time exception

Throwable does catch all exceptions at execution time

but your code has a runtime try/catch

as ghadi says, this is invalid syntax

if you wrote (/ 1 0) inside the try, it would be caught

Is there a way to catch read time exceptions?

if you're the one invoking the reader, sure

you haven't said in what context you're doing this

looks like slamhound plugin?

No. I’m just hacking in the REPL

the read is occurring in the repl

R!

you, as a user of the repl are not in the loop on that

you have to control the R if you want to catch read-time exceptions

another invalid thing in the same category is {:singlekey}

user=> {:singlekey}
Syntax error reading source at (REPL:6:13).
Map literal must contain an even number of forms

yeah. makes sense

My original problem had to do with lazy sequences. I figured it just now. Here is an illustration:

(defn read-numbers [nums]
    (try (map read-string nums)
         (catch Throwable e
           nil)))

  (first  (read-numbers ["1aa"]))

I guess it’s a common pitfall of lazy seqs

Any idea how to avoid this kind of issues?

force the map or move the try catch inside of it

don't use laziness?

or really in this case the "catching of bad input" is something that should be done around each read of the input, not around the whole job

(map #(try (read-string %) (catch Throwable e <something>)) nums)

I'd probably use mapv in situations like this @viebel

Alright folks, I've got a head scratcher. I have a map that I'm trying to serialize with Nippy. I think it's a map, at least. When I call Nippy's freeze method on my "map", I get this exception:

(taoensso.nippy/freeze
  (apply dissoc as (keys as)))
Execution error (ExceptionInfo) at taoensso.nippy/throw-unfreezable (nippy.clj:720).
Unfreezable type: class clojure.core$partial$fn__5839

(apply dissoc as (keys as))
=> {}

(type (apply dissoc as (keys as)))
=> clojure.lang.PersistentHashMap

(taoensso.nippy/freeze {})
=> #object["[B" 0x54ce414 "[B@54ce414"]

what makes you thing it is "just" a regular old map

The type call.

what makes you "think" its a map versus know its a map. where's the doubt coming from?

https://twitter.com/stuarthalloway/status/1149334853218967553

it's a gif that says "don't elide stacktraces"

Well, I've gotten bitten by the printer outputting something that looks like something normal but is actually something totally different. I don't think that's the case here.

I also know Datomic can output things in a somewhat unexpected type. Can't recall the exact instances. I don't think that applies here because the type is a PersistentHashMap.

Does nippy serialize meta?

I've tried restarting the REPL so there's no weird state.

Ooo, maybe.

That's gotta be it!

(meta (apply dissoc as (keys as)))
=>
#:clojure.core.protocols{nav #object[clojure.core$partial$fn__5839 0x7cfd1de5 "clojure.core$partial$fn__5839@7cfd1de5"]}

Yep, that's it:

(taoensso.nippy/freeze (with-meta (apply dissoc as (keys as)) {}))
=> #object["[B" 0x7cfeb74d "[B@7cfeb74d"]

seems like a weird default to serialize meta

Yeah, I agree.

I don't even see an obvious way to disable metadata serialization 😞

hi! I'm trying to read a clojure file forms using tools.reader/read, but it is failing because one form contains ::stc/opts keyword and that alias is defined with (alias 'stc 'clojure.spec.test.check) any ideas on how to read this file or how to configure tools reader to skip that form instead of throwing ?

the file I'm trying to read is https://github.com/clojure/spec.alpha/blob/0269b2cfefe4df7b68710decc94c623db4a6f630/src/main/clojure/clojure/spec/test/alpha.clj#L439

https://clojure.github.io/tools.reader/ use *alias-map*

yeah I'm using it, extracting it from ns declaration using tools.namespace, but don't know what to do about aliases defined with alias, I'm looking for a automated way for any file, not just for this file

you can't

it is one of the problems with using ::, its meaning is context dependent and the only sure fire way to have the context is to eval the file

hmmmm, will have to pre-parse the file looking for alias forms then

thx @hiredman

I could put a function call in a file that called a function in another namespace, which altered the alias for the current namespace, which would change how subsequent :: keywords are resolved

but I have a keyword with a alias

because clojure is a form at a time, a given toplevel form can be executed and effect the environment subsequent forms are read in

yeah

so for example, I could having something like (foo) in my file, in a namespace call it A, where foo is brought in from namespace B, and is defined like (defn foo [] (change-all-aliases-in-*ns*-to-point-to-clojure-core))

quite a hack

and clojure would happily load that file, and the aliases for keywords or symbols would be whatever before (foo) and after foo they would be all different

that is a pathological case, and I don't know what your intent is, I am just saying the bar for handling that completely generally is very high

yeah I'm doing clojure static code analysis and I'm struggling with this cases

it is very similar to the case of determining what the symbol foo/bar refers to in a given namespace, for the majority of cases you can figure it out just by looking at the ns form at the top of the file, but that is just due to convention and good taste

yeah it would be nice if we could tell the reader what to do when it can't resolve something

that is more or less what the alias map does

I was thinking more on common lisp condition system he

I believe the tools.reader code calls the alias-map as a function, which means you can rebind it to a function instead of a map

oh that is interesting

it is relying on a impl detail but it will work for what I'm trying to do, thx again @hiredman

Does anyone know if it is possible to load native libraries from the classpath?

no

by which I mean, I bet you could figure out some way to do it, but you will be so far off the beaten path that you would have to invent your own ffi to do it

usually if the native library is distributed in a jar you extract it somewhere (/tmp) and load it from there

That's kinda what I was thinking. It's a bit annoying because that means you need some custom startup script and everything can't be done within the JVM.

How about including the native libs on my classpath, copying them at runtime in my -main to a path on the java.library.path and then proceeding with app boot?

why do you need a custom startup script?

To do the "extract it" step.

looks like you got there, yeah, just do it in your app