"Other languages will be faster, but Odin attempts to be a "drop in and use" solution where the total run time of a query is not the top priority." Now I wonder what those faster languages/libs, since my main goal here is performance

Looks like I'm going to need to do some benchmarking...

@qqq there is a dyn var that controls recursion limits. Also, I'd recommend setting :gen-max on any colls in the recursion

@alexmiller what's killing me is not the recursion depth, but the breadth

like if a coll-of generates N items, and I have depth 4, I sudeenly have N^4, and if N is something like 100, then code dies

depending on what your logic program does it might easy to encode for a SAT or SMT

and there are really fast SAT solvers

ah, yeah, for some kind of query engine a sat solver won't likely work well

qqq so gen-max will help

The default is 20 I think but I usually set it to 3

we're doing things like putting large number of tuples (facts) into the logicdb, then running things like:

(run section-limit [v] (fresh [addr type vs sz n t] (artifact-data-section sha256 addr type vs sz) (artifact-data-section-entropy sha256 addr n t) (== v {:addr addr :type type :vs vs :sz sz :entropy n :entropy-types t})))

basically a: SELECT a b c d e f FROM artifact-data-section JON artifacat-data-section-entropy ON foo....

I need to dig deper into our use cases, but I think that's essentially what we're using it for...

looks like odin would get us the ability to do that --- I should benchmark core.logic vs odin for some typical things we do

tabling might help, it is memoziation or caching for relations, the other thing often done for performance is abandoning relational purity and use condu or conda

@alexmiller : :gen-max solved my issue; thanks!

If Clojure was minikanren , would clojure.spec generator be far more powerful? The main limitation to Clojure Spec seems to be "sometimes, you ahve to write your own generators to pass the 'such that' test" -- and in all examples, it seems like "if you could run the code backwards in a relational manner" -- then you would be able to automatically synthesize conforming examples. Thus, if all Spec code was limited to "minikanren like funcitons" insted of "arbitrary clojure code", it seems like spec / generators would be even more powerful.

And sooooper slow

only during the "automatically brute force test this function for me " phase right? it should not hurt runtime performance

Did you see the barliman talk at the conj?

@hiredman thanks, I'll read up on tabling

@alexbaranosky: datalog and spectr will be faste, but with a different set of tradeoffs

I think it's possible to JIT Odin via something like Truffle, but I'm still meditating on that one

Odin should be faster than core.logic done it used transducers instead of the list monad, but I haven't benchmarked it yet.

yeah, i think we need to let/help will byrd et al do another decade or two of research first 🙂

That being said, I have a query engine in production that uses a minikanren-esque engine, and the engine is rarely the bottleneck, it's network data loads, and out of order joins that are the biggest source of perf problems

what are you querying?

Diatomic data, but we want lazy query results and our datasets are too large for the heap, so Datalog wasn't working

And I'm on my phone so sorry for the type-os

interesting

Anyone else having trouble moving tickets in Jira from Column to column on active sprint?

@josh_tackett Wrong channel? Wrong Slack maybe? 🙂

haha no I actually was asking here @seancorfield Wondering if the problem is all of Jira or just our companies board?

Well, it's #off-topic at best for this channel.

@seancorfield Thank you for testing this for me!

How can I get lein to pass arguments to the underlying code? (I.e. I want to define a varaible inside my lein project.clj -- and then have mya ctual code be able to read that varaible and get its value).

@seancorfield not sure what you mean?

Comes through as regular messages via the IRC bridge. Phew!

I'm trying to create a project using [org.clojars.nicolaus/cljgae-template "0.2.5-SNAPSHOT"] as a template -- however, where do I put this depednency as I don't ahve a proejct.clj until I create the project ?

@geoffs @joshjones It puzzles me why my version is reasonably fast. I couldn’t come up with anything that would be significantly better in terms of performance.

@nooga: laziness (filter and remove resulting seqs are not realized) and, once you get past this, range seqs are specialized (depends on clojure version).

@cgrand but then, when the resulting vector of 2 lists gets realized, filter and remove still execute one after another I think?

=> (map realized? ((juxt (partial filter odd?) (partial remove odd?)) (range 5)))
(false false)

=> (let [[a b] ((juxt (partial filter odd?) (partial remove odd?)) (range 5))]
     (doall b)
     [(realized? a) (realized? b)])
[false true]

I see

so this would be an eager implementation then: (reduce (fn [[t f] x] (if (odd? x) [(conj t x) f] [t (conj f x)])) [] (range 10))

yes which is more or less group-by

I actually ended up rewriting my algorithm so that it just uses group-by instead of filter/`remove`. Thanks @cgrand !

For the record, using xforms: (x/transjuxt [(comp (filter odd?) (x/into [])) (comp (remove odd?) (x/into []))] (range 10)) more like your initial attempt or (x/into {} (x/by-key odd? (x/into [])) (range 10)) (like group-by but usually faster)

thanks, I stumbled upon xforms but haven’t tried yet 🙂

What’s the reason I have to call seq here: (shuffle (seq {:a 1 :b 2}))

I suspect shuffle is skeptical about the fact that you gave it an unordered collection

I assume nth has similar hang-ups, for example

shuffle works with java Collections, not clojure IPersistentCollections, and for some reason clojure maps don't implement the Collection interface

same with sets?

sets do implement Collection

yeah that is kindof odd. Although on my box: (shuffle #{1 2 3}) = [1 2 3]

So yeah...garbage in, garbage out.

I don't think that's GIGO

user=> (shuffle #{1 2 3})
[3 1 2]
user=> (shuffle #{1 2 3})
[2 3 1]

yeah, I ran it again

and it changed, but still I'm not sure what shuffle is supposed to do if something is unordered to begin with. Perhaps just return the argument? lol

an unordered collection is not necessarily something that will give you its elements in a random order

presumably it almost never is

it just gives them to you in a weird order

so if what you want is random, then shuffle would be a reasonable function to use

yeah, and considering shuffle is based off of random, I suppose it's fine.

(seq #{1 2 3})
=> (1 3 2)

hey folks, What is the right way to catch the control-c event ( or handling POSIX signals ) ?

yes, I wanted random, not weird 😉

@lxsameer https://github.com/amalloy/useful/blob/bcb07414cf3dd5a09794b76490c0cf18758f1888/src/flatland/useful/java.clj#L10

I can't seem to figure out why a multiline regex isn't working:

thanks

^ shouldn't return nil, right?

I think you also need the ?s flag

So the . matches the newlines.

yes! that did it, thank you

and there the documentation is, clearly stating I needed the 's' : /

@rauh un fortunately that code didn't help me

Hi, perhaps this is more of a programming question and less a clojure question, but I'm trying to copy a Ruby implementation to generate a signed URL for some API that doesn't have a clojure SDK. I see in their code they're calling SHA1.digest("xyz") and that returns something that looks like this "\x1F\xBE\xB7wtV<\xC6\xB6\xE7\xEC}\x19\xABA\xD2\x16\xDE\x94\x85". .Ruby also has a mehtod hexdigest that returns a regular string with no backslashes. I can't seem to find an sha1 library that returns a digest that looks like above, and furthermore i have the slightest knowledge of what any of this means. What is the above with backslashes?

@jdkealy that's the representation string of you digest

then what's hexdigest ?

hexdigest is the hexadecimal representation of you digest

for example

is there a clojure lib that will give me the string representation ?

you don't need that

if you have the digest already you can use it

in ruby they do Base64.urlsafe_encode64(Digest::SHA1.digest(to_sign + secret))[0,8]

i'm trying to achieve the same thing with pandect's sha1 algo

a google search for “clojure sha library” turned this up, not sure if it will be useful to you: https://github.com/tebeka/clj-digest and this: http://stackoverflow.com/questions/31729163/clojure-or-java-equivalent-to-rubys-hmac-hexdigest

@jdkealy you don't have to do the same thing

AFAIK Base64.urlsafe_encode64 method works with both digest and hexdigest

I’m doing some sha stuff right now and using org.apache.commons.codec.digest.DigestUtils/shaHex

which is from dep [commons-codec "1.7"]

with Spec, is there a way to specify a generator for a part of a spec when it depends on another part (e.g. when numbers need to sum to 1)

you probably need to specify a generator for the larger structure

of course

it'd be useful in this case for spec to have a helper for modifying the default generator for a spec, but I don't think it does

(in that case you could just pass it to gen/fmap and tweak the generated value however you like)

@lxsameer i copied the implementation and something's not quite right. i'm using clj-digest library

using this lib for b64 [clojure.data.codec.base64 :as b64]

the resulting "signed" variable looking like "s--N2E5M2Uy-" whereas in ruby it comes out looking as "s--epPiduos--"

@gfredericks ah yes fmap and such-that - cheers!

@jdkealy I'm not an expert on this matter but your function makes sense

@gfredericks spec has an option for that: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/spec

@tbaldridge How do you get the default generator though?

(s/gen :foo)

or maybe I don't understand the question

seems like a global override for the generator of a spec would be a bad idea

instead alias and give it a new generator

@jdkealy you are base64 encoding the hex encoding of the sha1 (or whatever string representation sha-1 returns) not the actual bytes of the sha1

@tbaldridge the problem is when using s/with-gen to define a new spec -- calling s/gen requires repeating the spec expression, which much be larger than :foo

A simple helper function would suffice.

It's not a huge deal

@gfredericks well you can create an alias (s/def ::foo ...) (s/def ::bar (s/spec ::foo ...newgen..)

Yeah. You might not want two names though

I’m parsing XML and encounter some invalid characters (while using clojure.data.xml). Is there a recommended way to strip these characters in clojure/Java? Maybe some commons library has a function for this?

@borkdude, like improperly escaped ampersands and such? I've had similar issues with html content inside of xml, and I've just "sanitized" via a potentially too-clever regex...

@geoffs there is an invalid unicode character in the input. I’ve discovered something like this in one of our dependencies though: (com.ximpleware.parser.XMLChar/isInvalidChar (int \a)), could use that

oh interesting. :thumbsup:

@borkdude Maybe you could do something like http://stackoverflow.com/a/14323524 (ported to clojure of course). Regexp it out.

@beppu I might use that if this function I found isn’t performant enough, thanks

Who is giving you invalid XML to start with? Maybe you could solve this problem earlier in the pipeline.

it’s coming from the internets

ha, the dreaded wire

regex is much faster

What does it mean when people react with a camel? 🙂

It was an allusion to Perl, (in)famous for its regexps.

http://akamaicovers.oreilly.com/images/9780596000271/cat.gif

Was just about to say.

Yes^

Aaaah 🙂

Hi everyone! I'm stuck trying to understand why the instance? isn't working as I expect. I have a function (error handler), which receives and error, and according to the type/class of the error, it do some work. I made a function which prints the type of the error, and also the result of applying instance? to that error object. This is the function:

And this is the result:

you don’t need (class e) - just use e

(instance? WebServiceException e)

Ufff... right. Thank you Alex.

I’ve been waiting for this for a while and finally it’s out: http://github.com/aphyr/dom-top specifically this: https://github.com/aphyr/dom-top/blob/master/src/dom_top/core.clj#L126 Figured that it needs a bigger audience

@dotemacs fcatch reminded me of try* from https://github.com/adambard/failjure

@beppu yea, good point. But all excited about letr 🙂 nifty

letr is cool.