Hello. Are there any libraries which calculate diff between two Clojure data strtuctures?

uhm, however: https://clojuredocs.org/clojure.data/diff

this seems weird:

(merge nil "")     ; => {}
(merge "" nil)     ; => error, string cannot be cast
(merge "" "")      ; => error, string cannot be cast
(merge nil "asdf") ; => error, char cannot be cast

why does the first work?

@andrewboltachev there's been some recent discussion about it. diff is a little slow. Some old solutions have some tradeoffs: https://github.com/brentonashworth/clj-diff https://github.com/brentonashworth/clj-diff

thanks @john !

And https://gitlab.com/robin.heggelund/differ, https://github.com/GeorgeJahad/difform and https://github.com/AlexBaranosky/gui-diff,, which looks the most recent

no prob!

I mean, differ looks the most recent

yes good stuff. My task is a little modest at a time being, but I'd be happy to join a disussion in the future as well

Is there a core function or library out there do this? Kind of an extension of cond-> and cond->> where data is available in “test” as well? Something like this

(cond*-> {:a 1 :c 3}
  #(= (:a %) 1)  (assoc :a 10)
  #(= (:a %) 10) (update :a + 10)
  :b             (update :b + 5)
  :c             (update :c + 8))

;=> {:a 20 :c 11}

The most intriguing solution I've seen actually leverages Clojure's MVVC persistent data structures, from this gist: https://gist.github.com/danjohansson/add5515b2067b3036044d450cbec08f3 but I was never able to get it to work.

@tap a common solution is to put a thrush (thread) within a thrush, if you're willing to do that.

@tap for instance,

(-> []
    (conj "first" "second")
    (->> (mapv str/upper-case))
    (conj "last"))

That example was taken from Howard M. Lewis Ship's awesome recent blog post on thread macros here: https://medium.com/@hlship/confessions-of-a-threading-macro-addict-5a026dae4af7

@john Interesting, thanks.

So I can do something like this

(as-> {:a 1 :c 3} $
  (cond-> $ (= (:a $) 1)  (assoc :a 10))
  (cond-> $ (= (:a $) 10) (update :a + 10))
  (cond-> $ (:b $)        (update :b + 5))
  (cond-> $ (:c $)        (update :c + 8)))

It's getting there

as-> is probably the most powerful.

all those cond->s look somewhat redundant, though, and to my eyes actually detract from from the readability of the form, which is the whole point of thrushes anyway.

@tap We wrote our own condp-> and condp->> macros that thread through the predicate as well as the expression.

@josh.freckleton did you get your answer?

(when (some identity maps)
    (reduce1 #(conj (or %1 {}) %2) maps))

i think its a coercion on ""

but this is conj: ([coll x] (. clojure.lang.RT (conj coll x))) and i'm not sure how to follow it

ok, that looks like it means

static public IPersistentCollection conj(IPersistentCollection coll, Object x){
	if(coll == null)
		return new PersistentList(x);
	return coll.cons(x);
}

from RT.java. and IPersistentVector:cons will coerce x to a map$Entry

and in APersistentMap, cons dictates that if its not a mapentry, or vector, try to seq it and then cast the first thing to a map entry and there's your kaboom

so the final answer is (seq "") = nil which is a valid map entry

does anyone know what RT stands for in clojure.lang.RT?

I think it's run time

ah thanks

Yes

I load and parse a xml file with clojure.data.xml. Unfortunately the files linebreaks between the tags are also in the resulting structure and are recognised as nodes. How can I filter the linebreaks?

@danielgrosse sounds like something StackOverflow may be able to answer

I have a potentially noobie question. I'm using thinktopic/lazy-map to cache various levels in a data structure which eventually reads things from disk when not cached. I have a function which returns an instance of the lazy map where when you access the map values they are realized, the disk is read (on first access) and values are returned. I would now like to find a way to make the returned map sorted by keys but retain the laziness of the thinktopic/lazy-map implementation. Just doing an (into (sorted-map) lazy-map) forces evaluation of the values and looses the laziness as all values are then realized.

never mind, think I figured it out

I'm trying to connect to sqlserver and can't fix "no suitable driver found for jdbc:sqlserver" I've tried downloading the sqljdbc42 jar file and placing it in a half dozen different folders but I'm just kinda lost on this being a C# dev. Anyone be able to give me some pointers?

if it helps here is the diff of my connection attempt. https://bitbucket.org/matthewdaniel/leaderboard/commits/bcebac3f9bcf4da46dd575c4fa072fb09b393851

… this is not actually referring to core.spec, I use a parameter in a lot of methods called spec

What does the file look like?

Or what does your ns declaration look like?

Hm, strange. That looks fine.

found it

I tried commenting out an fn with #_

and paredit did something screwy and left me with unbalanced parens

So I’m guessing you get the Line 1/Column 1 error when you have an extra ) somewhere

I feel like the error message could be better in this case, but I know not how hard that would be

or rather, in this case it tried to resolve a symbol as if it were at the highest level of the ns

ok, maybe detecting that is harder than I thought

it basically saw this at the top-level: [spec {:keys [params error] :as vresp}]

but it eventually would have parsed a closing ) with no matching (

still, why did it say 1:1? Why not give me this line number?

I think it's to do with how the reader works in clojure, it doesn't quite store enough metadata.

@cpmcdaniel Maybe this is something joker could help with. Or perhaps it’s worth raising a JIRA issue for a better exception.

Wondered if I could get some thoughts from the community about this PR https://github.com/sjl/metrics-clojure/pull/120 Basic idea is to get metrics per endpoint, but now I'm trying to integrate it into my own app, I'm finding it a little bit lacking. It's easy enough to wrap each route, but our application routes are collected into a single handler by compojure, and then a list of middleware is applied to that single handler. If I wrapped each one, the metrics would not record the time taken for global things like wrap-params, wrap-auth etc. I could refactor our app to accommodate but this seems like a reasonable pattern.

I'm having some trouble with a spec under stest/check, I think I may be using s/and incorrectly `

(s/fdef between
  :args (s/or :part (s/cat :a ::id :b ::id)
              :full (s/and (s/cat :choose ::choose :a ::id :b ::id)
                           #(not (pos? (compare-ids (:a %) (:b %))))))
  :ret  (s/nilable ::id)
  :fn   (fn [{:keys [ret args]}]
          (let [{:keys [a b]} (get args 1)]
            (between-ok? a b ret))))

it's generating failures where the :choose :a and 😛 are fine, but where a and b are out of order, and so fail the not pos? compare-ids predicate

my understanding is that s/and should generate lists of those arguments and then filter out the ones that fail the second test

and, naturally, as soon as I ask the question I figured it out! the :part branch is a 2 arity on version of the between function, which just provides a default :choose. the ordered? predicate has to go on both the :part and :full branches of the or

so

:args (s/or :part (s/and (s/cat :a ::id :b ::id) ordered?)
              :full (s/and (s/cat :choose ::choose :a ::id :b ::id) ordered?))

I suppose that's one of the reasons people like their routers to be just data.

what's the best intro talk to spec?

I just watched this one last night:

https://vimeo.com/195711510

It's almost two hours, tough. If you want something shorter, I don't know one.

great, thanks!

I'm wondering a bit about how to really use spec in a project - do specs go in their own namespaces or in the namespaces of the things they're speccing? I understand it's programmer choice to a degree, but is there a best practice?

Are there consequences between one or the other?

I've found the main consideration there to be cider support. jump to definition for specs doesn't exist yet

it's made me a little happier with specs in their own namespace, just because they're a little hard to find when you've got fspec and defn mixed in together

Do you usually group all the specs together in, like, a spec directory, like we do now with test?

is there any way to type-hint something in a threading macro?

e.g.:

(-> x
  .somefn
  .someOtherFn
  (#(.getString ^MyAnnotation %)))

^ this works, obviously, but looks ugly

Maybe a let and pass the fn in?

(let [fun #(.getString ^MyAnnotation %)]
 (-> x
   .somefn
    .someOtherFn
    fun))

GAHHHH

its good enough

not really achieving anything more than what I have ¯\(ツ)/¯

I’m refactoring a part of the app that created a closure and then returned an anonymous function accessing data in the closure. To aid in refactoring I’m adding an arity that exposes all the data in the closure. Is it a total hack to just leave that around afterward for convenience?

maybe replace the closure with a defrecord that implements IFn, or a map with all the data and an entry with the function

oh - that’s much nicer

implementing IFn properly would mean it would still work while transitioning, thanks for the idea

@captainlexington I'm still experimenting with layout, but right now I've got a spec namespace in each module mixed in with src. a top level spec namespace might be nice, though

Mmmm

Question: Is it safe to use in-ns inside a macro to resolve functions defined in a different namspace.

(defmacro load-data
  [filename]
  (in-ns 'other-ns)
  (cons `do (for [k# (seq (file filename))]
              `(intern 'other-ns
                       '~(symbol (name (first k#)))
                       (into {} ~(second k#))))))

don’t change namespaces to resolve things

it sounds like you aren't just reading in the file, you are reading it in then sending it through the compiler

yeah, that macro splices a function call to a function name read out of a file

the compiler resolves symbols and compiles functions calls according to how clojure the language works

if you read it in as data you can interpret it however you wish doing whatever with it

okay. right I am reading in the file and compiling the data. i need to use the functions defined in the file.

wait, defined or referenced?

sorry referenced*

my inclination is to use a weaker construct if possible

for example, instead of emitting a call to the function in the line of code, you could emit a call to a function that explicitly looks for the definition of that name in the target namespace

the resolve function would help here

or ns-resolve since you know an explicit namespace you want to do the lookup in

@noisesmith are you talking about this line? (into {} ~(second k#))

no

Can anyone give me a hint about this: https://stackoverflow.com/questions/44315467/clojure-macro-perform-action-only-if-a-symbol-is-defined ? 😬

'~(symbol (name (first k#)) - this turns a string into a function and calls it

@nvtf for starters putting a call to eval inside a macro is a sign you are making a mistake, almost always

@dhruv you can replace that with a call to ns-resolve with the target ns, which returns the var corresponding to that string (if found)

yep, only "ugly hack" that came to mind though

@noisesmith thank you. let me try that out

ns-resolve takes two symbols and returns a var, you can deref the var, or for a function, just call it

ah!

@nvtf amongst other reasons not to use eval, eval doesn’t use values in your local scope, it only resolves at a namespace level

is there anything that resolves in local scope? (`resolve` seems to work that way as well)

the right way to provide values at runtime is to create a function and supply them as arguments

in fact, apart from wanting to intern a value with the metadata that defn creates, I don’t see any reason for that code to be a macro

@robert-stuttaford did you guys switch from Yeller to something else? i just noticed the blog post saying they were shutting down last year.

@devth sentry 🙂 works great

cool, thanks. that's what we're looking at too 🙂

Hi all, how efficiently write big hash-map into edn file? I’ve tried pprint/write db :stream out - but it is slow 1.2M in 12 seconds 😞

pprint is slow

Have you tried plain prn?

also, it’s an extra dep, but transit is super fast and super reliable in my experience, and handles a superset of edn

@ghadi - yep it’s faster - 100ms 🙂

is that a difference of java in memory writing whole map vs stream writer?

It's probably the difference between pretty-printing and printing unformatted

Presumably 1.2 MB edn file isn't meant for human consumption, so pretty-printing isn't important?

Been bitten in production by this today: (into (sorted-map-by (constantly 0)) {:c 1 :b 2 :a 3}) ==> {:c 3} Ok, we didn't use (constantly 0) as our comparator but a bug made it similar for some values. Even so I would never expect that a function accepting a map and returning an equivalent one with its keys sorted would actually change the number of keys and mix what values were assigned to which keys - i.e. mangling the map. I've now read the clojure doc on avoiding comparator mistakes. Is this just a case of "Buyer beware"?

can you have a fulltext index as well as AVET index on an attribute?

I haven’t tried yet, just curious before I give it a shot--the docs don’t seem to suggest it’s not possible, based on what I’ve read so far

whoops, maybe I should have asked in #datomic

@apbleonard the documentation there certainly seems lacking, but yeah seems to me that's a consequence of how sorted maps work, the comparator needs to specify an absolute order

as assoc etc do all of their key comparison via this function https://github.com/clojure/clojure/blob/clojure-1.9.0-alpha14/src/jvm/clojure/lang/PersistentTreeMap.java#L325 which defers to the comparator. there's no logic for "ok the comparator says they're the same but are they really the same?"

the comparer function is how it tells things apart

you can get a constraint violation if you do the opposite, feed it a function that lies and says everything is different

user> (into (sorted-map-by (constantly 1)) {:c 1 :b 2 :a 3})
{:c 1, :b 2, :a 3}
user> (into (sorted-map-by (constantly -1)) {:c 1 :b 2 :a 3})
{:a 3, :b 2, :c 1}
user> (into (sorted-map-by (constantly -1)) [[:a 1] [:a 2] [:a 3]])
{:a 3, :a 2, :a 1}

Has anyone evaluated the different DB migrations libraries (eg ragtime, drift, migratus) and formed strong opinions on any? It's hard to understand the actual differences and rationale behind so many similar libs

what are some good Clojure test suites I can look at for inspiration?

I’m coming from an RSpec background and I would venture that the “best practices” in Clojure testing are way different

also, what are peoples experiences with https://github.com/slagyr/speclj? is it worth giving a shot?

@schmee my humble opinion is that you're best to stick to clojure.test with as few other dependencies as possible. that way you get the best tooling support, helpers when you need them, etc etc. clojure.test isn't very pretty, and can be fairly awkward, but it's worth just dealing with that to be using what everyone else is using here (your test framework shouldn't be what you distinguish yourself with)

my background for that is rails+rspec+cucumber -> rails+test::unit -> clojure+midje -> clojure+clojure.test

there might have been another ruby testing lib in there at some point, I forget

fwiw speclj was sort of an rspec port. it feels really comfortable for me (coming from rspec) and i like the auto-runner a lot, but i’m not trustworthy, as i’ve contributed to it and have been using it for awhile.

I use clojure.test and venantius/ultra for readable output, may try midje one day

an example here is the cursive+clojure.test integration at this point is freakin amazing, I'm not sure if such is true for the other things

afaik test.check only has clojure.test integration thus far

Using cursive and clojure.test, didnt even know integration existed...

really? it's pretty incredible. go to a test file (while you have a repl session running) and hit your intellij "run tests in file" key combo. try breaking one of the tests and see the data diff you get in the sidebar etc

@trptcolin any issues in particular you’ve encountered with Speclj coming from RSpec?

Yes, I'm reading the docs...amazing, I'm relatively new to Cursive...was using SublimeText with a REPL...never ever looking back

@apbleonard comparators are so easy to screw up that there's a dedicated guide to them: https://clojure.org/guides/comparators -- on the surface they seem so easy, but there are gotchas related to total ordering that are not intuitive

@bfabry @joelsanchez Improvements coming to that soon too, hopefully next week!

@cfleming Hey, how can I tell Cursive to indent hiccup with two spaces? Couldn't find anything about it

Currently under discussion here: https://groups.google.com/d/topic/cursive/OM4T_nPGkfA/discussion, feedback welcome, either on the ML or over in #cursive

@joelsanchez There’s no way to do that right now that I can think of - vectors are always considered data and indented one space.

@cfleming Guess I can live with that

Hiccup is an interesting one - there are a few things that would be nice to do with it, but it would always rely on some heuristic to identify it.

Frankly I'd like to have two-space indent everywhere, where do people find one-space indentation more readable?

The problem is that this would indent your let blocks like this:

(let [a 1
       b 2]
  ...)

Yikes

What about "indent data with two spaces except in let blocks and other exceptions I can setup"?

It would probably do all sorts of horrible things to e.g. datomic queries, but I’m not familiar enough with them to cook up an example.

I think it would be a lot of exceptions.

I give up 🙂 better this way indeed

I’m afraid that one is going to be a living-with-it sort of situation 🙂

Can't have the indentation and eat it too

It’s the dark side of “everything is just data”

Yeah, that wasn't a very good idea after all, going back to PHP

Am I using clojure.pprint/*print-miser-width* wrong? The following seems to have no effect:

(binding [pp/*print-miser-width* 1000]
  (pp/pprint …))

@lvh what about *print-right-margin*

@noisesmith that did it; thanks!

there doesn’t seem to be an incantation that makes it spread a long vector across two lines

(binding [pp/*print-right-margin* 120 pp/*print-miser-width* 100]
  (pp/pprint (range 100)))