@drewverlee to me, clojure.spec and plumatic/schema are comparable in terms of what they do, that is specification, validation and then data generation (actually both piggiebacking on test.check

I used to use schema and apart from a few quirks I found it very nice, especially because it was data oriented (specs are maps)

clojure.spec has a couple of differences but still works quite nicely, and I opted to use it for my newer projects (together of course, with the team I am working with) mainly because it is going to be in the language "core" and therefore directly maintained

about clojure.spec some function does not accept pure data it seems (found a couple of example macro that does not accept data), but probably things are going to change (it is still alpha code)

I have never used scjsv but the big difference I see is that it does validation of the data input your receive "from the wire" as JSON

schema and spec can do both validation of data "from the wire" (after conversion) and "runtime" data

while the program is running

thanks @richiardiandrea. I think part of what im confused about is if something like protocal buffers (by google) are only concerned with encoding & decoding or if they can play a similar role in data validation. It seems like the answer is yes. But obviously you can only take advantage of this if the sender has decided to offer the option.

I think the offer some kind of validation yes, because when you cannot de-serialize correctly they throw. I am not very familiar with protobuffer too much and I would investigate if there is in-depth error detailing for instance, which is actually a big thing in clojure.spec (each data piece is reported together with the error)

Hey, hivemind

(continuous-take-while #(not= % "END") ["START" "MIDDLE" "END" "START" "MIDDLE" "END"])
;;=> [["START" "MIDDLE"] ["START" "MIDDLE”]]

@lambeta loop with a take-while and skip using the count from the last collection you take-whiled?

(->> ["START" "MIDDLE" "END" "START" "MIDDLE" "END"] (partition-by #{"END"}) (partition 2) (map first))

@lambeta is this close enough to what you want? (remove #(= % ["END"]) (partition-by #(not= % "END") ["START" "MIDDLE" "END" "START" "MIDDLE" "END"]))

I think the function is underspecified

Note that my solution will break if you have two consecutive instances of ”END”

Combining my solution with @danboykis’s we get

(->> ["START" "MIDDLE" "END" "START" "MIDDLE" "END"] (partition-by #{"END"}) (remove #{[“END”]}))

This is more robust.

(constantly [["START" "MIDDLE"] ["START" "MIDDLE"]]) ;; also works

I like gfredericks' solution if that's all you want

@danboykis I implemented it with this solution. but just think it’s a little ugly. because we need remove the “[END]"

@gfredericks that’s hard code, right?

yep

I doubt it's what you wanted but my point is you weren't very specific about what you wanted because you didn't explain what sort of variety there is in the input

or how the input and output relate

oh wait I didn't read closely enough

@lambeta you can write a custom loop/recur that wouldn't remove, but it's not as short

I somehow misread and thought you had only given the input and output :)

sure.

also I assumed continuous as part of the name implied you're working on potentially infinite sequences

one second, let me prepare the whole problem.

I have a file named molecules like this:

COMPND      AMMONIA
ATOM      1  N  0.257  -0.363   0.000
ATOM      2  H  0.257   0.727   0.000
ATOM      3  H  0.771  -0.727   0.890
ATOM      4  H  0.771  -0.727  -0.890
END
COMPND      METHANOL
ATOM      1  C  -0.748  -0.015   0.024
ATOM      2  O  0.558   0.420  -0.278
ATOM      3  H  -1.293  -0.202  -0.901
ATOM      4  H  -1.263   0.754   0.600
ATOM      5  H  -0.699  -0.934   0.609
ATOM      6  H  0.716   1.404   0.137
END

(("AMMONIA" ("N" "0.257" "-0.363" "0.000") ("H" "0.257" "0.727" "0.000") ("H" "0.771" "-0.727" "0.890") ("H" "0.771" "-0.727" "-0.890")) ("METHANOL" ("C" "-0.748" "-0.015" "0.024") ("O" "0.558" "0.420" "\
-0.278") ("H" "-1.293" "-0.202" "-0.901") ("H" "-1.263" "0.754" "0.600") ("H" "-0.699" "-0.934" "0.609") ("H" "0.716" "1.404" "0.137")))

this is my solution:

(defn read-all-molecules [input-file]
  (map (fn [molecules]
         (let [[_ name] (str/split (first molecules) #"\s+")
               atoms (map (comp (fn [[_ _ & type-coordinates]]
                                  type-coordinates)
                                     #(str/split % #"\s+"))
                               (rest molecules))]
           (concat [name] atoms)))
       (filter #(not= % ["END"])
               (partition-by #(= % "END") (line-seq (io/reader input-file))))))

@lambeta personally I would divide the mapping in two, the first identifying a "compound" and then an "atom". It seems to me it would simplify the solution a bit

{:compound […] :atoms […]}

maybe similarly to a two-way parsing...the output can be the above or [{:compound {:atoms []} {:compound {:atoms []}}] depending on your requirements

I was thinking the most natural way to represent it would be

[{:compound “AMMONIA” :atoms [{:atom “N” :x 0.257 :y -0.363 :z 0.000} …]} …]

You want to pick the data structure that will be the easiest to understand and manipulate, in general.

Hi All, I am new to Clara rules. One of my usecase I supposed to iterate list of object and perform validations in clojure. Anyone could you please suggest an approach, how it can be done?

@lambeta Specter has this functionality built-in:

(select (continuous-subseqs #(not= "END" %))  ["START" "MIDDLE" "END" "START" "MIDDLE" "END"])
;; => [["START" "MIDDLE"] ["START" "MIDDLE"]]

Wow, cool

can also do transformations with it:

(transform (continuous-subseqs #(not= "END" %))
  (fn [v] [(count v)])
  ["START" "MIDDLE" "END" "START" "MIDDLE" "END"])
;;=> [2 "END" 2 "END"]

I've been noodling on this and feel unsatisfied with some of the approaches I've tried … I want to go from [{:a 1 :b 2} {:a 3 :b 4 :c 5}] to {:a [1 3] :b [2 4] :c [5]}

(merge-with (comp conj vector) {:a 1 :b 2} {:a 3 :b 4 :c 5})

except that left 5 not in a vector

yah that's one of the wrinkles

I thought maybe reduce-kv to set- or vec-ize the values first

I started to rathole on trying to minimize the number of steps 🙂

oh and that doesn't build a flat vector or set of values, which I'd also like: (comp conj vector) {:a 1 :b 2} {:a 3 :b 4 :c 5} {:a 6 :b 7 :c 8}) => {:a [[1 3] 6], :b [[2 4] 7], :c [5 8]}

instead of :a [1 3 6] etc

(merge-with list {:a 1 :b 2} {:a 3 :b 4 :c 5})

ah that puts me on the right track ... (merge-with (comp flatten list) [{:a 1 :b 2} {:a 3 :b 4 :c 5} {:a 6 :b 7 :c 8}])

=> {:a (1 3 6), :b (2 4 7), :c (5 8)}

i'm not sure why this doesn't work actually

i think its blowing up on me making a vec from v, but it seems right to me

ah because (vec 5) blows up

(vector 5) works

I still feel like flatten shouldn't be required … this still feels like it should be a one-pass fold

@hagmonk: does your function only take two maps, or can it take arbitrarily many?

oh I see, there's 3 above

arbitrarily many … same keys in each map though

(merge-with (comp conj vector) {:a 1} {:a 2})

but that requires the flatten

ha don't overthink it. if its a vector, conj it, other wise make a vector of the two

@dpsutton: there has to be a final pass where keys that are not in collections get put into vectors, as well

something like (into {} (map (fn [[k v]] [k (if (vector? v) v (vector v))]) merged-maps))

get all the keys from all maps, and reduce over each key, making a vector of all of the values

although you could argue that getting all of the keys across all maps is double enumerating

Is it possible to mix a protocol with a multi-method?

Like I want to extend a type, but I want different implementations for each combination of type/first-argument

if you know the keys in advance (which I do in this case)

Why is threading macro not working with deref?

(-> uri @(http/get))

Because deref magically adds extra parentheses and so changes what the threading macro does

i see. it works if i separately use (deref)

Try using the deref function instead and it should be less tricky

hi guys

is there way a good way to generate sentence from some words, which will discover the relations between them, for example:

#{Alice shoe train} -> maybe generate: alice wear shoe in the train ?

abdullahibra I think it's more an algorithmic/semantic issue than a clojure issue

yeah @abdullahibra that’s some advanced linguistics and/or machine learning you’d need 🙂

You can also try to dive in Natual Language Processing abdullahibra 🙂

okay great guys

@yonatanel see what that desugars to:

user=> (clojure.pprint/pprint (clojure.walk/macroexpand-all '(-> uri @(http/get))))
(clojure.core/deref uri (http/get))

Has anyone compared or experienced core.async vs manifold vs pulsar?

@yonatanel All three are great I think, each one has it's strengths. I use all three in one app - they go together nicely and are supplementary in many aspects. Just starting though, so I can't speak about stability and how things will turn out long term.

@andreas-thoelke What are your reasons to use each one?

i've been reading over the manifold source and the ability to treat channels as streams and seq's seems pretty nice

and splicing things into each other, etc

@yonatanel If you want to model complex async processes, Pulsar actors allow you to do this very easily (partly due to the good documentation and core.match being integrated). I find doing similar things in Manifold more abstract/challenging/ takes more time to learn/ has fewer examples/docs.

@andreas-thoelke Good to know. Would you migrate everything to Pulsar or do you have particular reasons to use core.async and manifold?

@yonatanel core.async is more about connecting parts of your system, rather then a general async/event driven lib. I think Manifold is very elegant when it comes to chaining deferreds/promises and combining and inspecting stream graphs.

Pulsar is very feature rich (it basically has reactive programming and core.async included), it's still pretty exotic in the community, so you'll probably only want to buy into Pulsar if you want actors.

Hello all, WTF is that ?

(slurp "temp\2meses.csv")

this is just a text file

@fabrao it has to do with the backslash in your string. If you want a literal backslash in your string you have to escape it: "temp\\2meses.csv", otherwise java thinks what follows is an octal number representing a char. That's why it says 'Invalid digit: m', m is not a valid digit in an octal number.

Ouch! I´m dumb as* rs, sorry !!!

Hello everyone, looking for devs in the Edmonton area who may be interested in a clj meetup.

@flyboarder remote link?

I mean will it be streamed?

@andrei: depends on the venue

We are working with Startup Edmonton to use their space or a partner

one sec, where is edmonton?

in Finland?

or Canada

Canada

That would be Alberta, Canada, yes?

Yes @nkraft

nvm what I said before, I confused channels :LOL:

erg, if I have some sequence of stuff like [“a” "b” “c” “d”] how do I say set each of these elements to true in a map?

@mnewhook my implementation is pretty naive:

(reduce (fn [m k] (assoc m k true)) your-map ["a" "b" "c" "d"])

arg, ok I was trying reduce-kv. Yeah that works 🙂 Thanks!

Like this?

(zipmap ["a" "b" "c" "d"] (repeat true))

@flyboarder: maybe you are already aware of it, but there is also a #C0736RLDT channel

It's like the #clojure channel except more polite.

@stand: lol like #clojure but we apologize for posting 😂

(def str (fn [& args] (apply clojure.core/str (conj args "Sorry, "))))

does anyone here know of strategies for maintaining relationships between caches? e.g. let's say I cache some data and compute some roll-up data based off it. I want properties like automatic, efficient re-calculation of the roll-up should any origin cache data change. sort of like a materialized view.

it feels like a very well-worn area but I'm not finding a lot of papers or libraries that talk about techniques, either in clojure or java

I think generally the solution would be to cache one or the other

I actually have a plan to build a library around this

not that it helps you in the least right now 🙂

heh, not immediately anyhow!

I tend to build a cache invalidation system, where caches can depend on multiple keys, then you can touch any of them to invalidate anything that depends on them

then you have a background process that checks the cached vs real value for the computation, to make sure you didn't miss anything

as a general solution, it would be hard to beat

that's basically it, in a nutshell

so I had this idea that you could register a function that takes a cache entry and maps it to a key, and another function that maps it to a value

so for an origin cache, one function represents the relationship with the "downstream" cache key, and the other function can be used to recompute the downstream cache value

and maybe that JCache listeners or events would be a neat way to glue things together

but then I got that "am I reinventing the wheel?" feeling …

Is there a way to see through wats going on under my lein build? A verbose log or sorts ?

there is the DEBUG env variable

Lein did set my hair on fire today.. Have a few java files to compile and have the right :java-source-paths configured in project.clj... yet it doesn't compile any of the files...

I would work backwards from there

what makes you think it isn't compiling the files?

checked the target folder...

and of course imports fail with classnotfound

what tasks have you run since you added that to lein?

This used to be working until today.. I have just removed a few libs today from project clj ... and stopped working (dont see any correlation obv).. tried lein clean and a lein repl

lein compile

@hiredman does :aot :all influence java classes like this?

lein javac is specifically the task that compiles java source, but it is setup to run before a few other built in tasks like compile and jar

I would sort of expect it to run before lein repl too

are you using any plugins?

I was expecting it to run before lein repl ... anyway just tried lein compile with no luck

if you are using any plugins, you should disable them all, one of them could be fiddling with whatever, you should also double check your project.clj

just cljsbuild plugin

roll back to a previous version, and see if that does in fact build the classfiles, if lein clean isn't a regular thing you do, you could have broken compiling a while back and been using stale class files in target/ since then

definitely try disabling cljsbuild

yeah, will be doing just that...

I'd try lein new ... and try to get a hello-world java class compiling by itself

even better if it's a known working example off github or whatever. if you can't get that working it's your local environ

if there was some issue with that, like a missing jdk or whatever, then lein would complain

Haven't done clean for weeks... so yeah I might have been working on a stale set... might have broken in between..

does anyone know why my app might see a suitable driver for microsoft sql server when run from lein ring server but not from tomcat8? running from command line works on the same server

if you've never played with git bisect, it is very cool

I have not... @hiredman , thanks for the suggestions.. let me narrow it down..

@dpsutton in similar cases I've looked at the classpath in the process to verify they're the same

i'm not specifiying a classpath for either. They both list their dependencies and can load them all

I mean the classpath that lein provides when it invokes the jvm for you

but i can run it on fedora 24, windows, but not ubuntu 16.04

ah. how would i go about finding that?

we are a .NET shop so java is half familiar, half completely foreign to me

tomcat may do something weird with classloaders, so whatever classloader ends up trying to load the class can't find it, you may need to explicitly load it from some other classloader

especially with tooling and setup

I'd use VisualVM

the strange thing is that we were running everything fine under tomcat7/14.04/java7 but recently updated to java8/16.04/tomcat8 and all has gone to hell

I'm pretty sure you can attach and get classpath from there, and it will be much nicer than grepping process lists

try something like (.loadClass (.getClassLoader clojure.lang.RT) "whateverthedriversclassnameis") at the top of your source file

com.microsoft.sqlserver.jdbc.SQLServerDriver

(.loadClass (.getClassLoader clojure.lang.RT) "com.microsoft.sqlserver.jdbc.SQLServerDriver")?

yeah, something like that, that might not be the correct classloader method

hmm, no such luck

i ran it from lein on the server, verified it worked, then compiled it there and moved it to a directory so tomcat could see it, and no luck

and in the webapps folder, in the lib i've got sqljdbc4-3.0.jar, which should be all that i need

you can put jars directly on to the classpath if necessary

@hiredman , I figured it , but its puzzling.. I have a dev profile which adds 'dev' folder (containing user.clj) to the :source-paths... If I remove :source-paths from dev profile, everything works fine.. java files do compile... if I add it back, it doesn't compile any of the java files.. any thoughts ?

I’m looking to read an image file from disk and store it in a database. Does anyone have a quick link on the best way to do this (hopefully just using `http://clojure.java.io)?

Storage isn’t the issue. I’m just wondering the most idiomatic way to read a binary file.

@akiva slurp is easy

though not the most efficient

cdine: the dev profile is overwriting the whatever the default profile is

instead of merging

@danielcompton, ah, I was under the impression that slurp only handles strings.

slurp handles many many things

Well, let’s give it a go.

slurp returns a string and goes through a reader, it is not appropriate for binary data

oh returns a string yes

use a ByteArrayOutputStream

copy an input stream in to it, call toByteArray on it

guess so.. But its supposed to merge.. at least as per the docs...

cdine: the exact thing lein does to combine profiles is kind of complicated

https://github.com/technomancy/leiningen/blob/master/doc/PROFILES.md#merging

thanks..

Thanks, @hiredman.

This is just a weird statement to me: "Latest stable release: 0.6.2-alpha3"

It sure is

just throwing this out there, but this doesn't look like an invalid connection string for sql server, does it?

java.sql.SQLException: No suitable driver found for jdbc:sqlserver://[servername];database=[database];user=[user];password=[password]

the jdbc:sqlserver:// prefix isn't invalid by any chance is it?

i'm just real sick of this issue and grasping at straws here

god I hate this stuff, struggled with it too

worst part is everything worked until an upgrade

and it works on all machines except when run through tomcat

are you using a connection pooler like Hikari or just straight up connection?

straight up

just an internal tool for us 5 developers

(aside: connection pooler stuff is kinda cool for enqueuing work against the DB, not just load, I was sold after using Hikari CP)

you're using com.microsoft.sqlserver.jdbc.SQLServerDataSource?

[com.microsoft/sqljdbc4 "3.0"]

as the 3.0 version is on maven so i don't need to keep it locally

and using clojure.java.jdbc

yeah

basically, clojure.java.jdbc/get-connection is where all the magic happens

java.sql.DriverManager/getConnection is passed the string you supply as a URI

so you could take a step back and try some non-URI connection parameters as specified in the doc string for get-connection.

problem is that this hasn't changed. only the upgrade java 7 -> 8, ubuntu 14.04 -> 16.04

but i was thinking of changing over to the map style

just to see if that worked

could java 7 -> 8 have rolled java.sql.DriverManager? if so that could definitely affect it

but i'm running java 1.8 on windows and fedora 24 and both work

what do you mean rolled java sql DriverManager?

I'm not sure if that library is pulled in as a transitive dependency by clojure.java.jdbc or whether it's part of the JDK itself

if the latter, bumping java versions could have changed it and caused it to break

is there a way to restore it then? or to check to see if this is indeed the issue?

and thanks so much for all of your help by the way

oh no worries, I totally tore my hair out getting pgjdbc-ng to work and learned more about this than I wanted to know

so here's another wrinkle. I rolled java backl to version 7 and everything works again

hmm

like without doing anything. tomcat7 points to the same war file haha

oh java. this is crazy talk

this whole java.sql.DriverManager thing is basically taking a string and then trying to figure out "I wonder if I have a class somewhere I can reach that understands this kind of connection?"

so I feel it's sensitive to deployment environment, config, and platform changes

yeah. that seems crazy to me that its a runtime check

how could you every really be comfortable

its crazy to me that i'm not saying sql server dep and at compile or link time its saying all of this is present and accounted for

well it's not only just present, but that also the driver groks the rest of the connection args you've passed in as params

so being explicit with a map at least lets you rule out some uncertainty to begin with

https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc.clj#L197

appreciated

got it back online with java 7 and i'll try some more stuff tomorrow to see if i can get it working with java 8

@dpsutton: have you tried the other sq server driver? [net.sourceforge.jtds/jtds "1.3.1"]? Not sure it's even remotely related.

again thanks so much for your time

can it talk to microsoft sql server?

if so i'm interested

but sean corfield mentions he only tests against 4.0 of microsoft's driver, which they make you download

rather than distribute

hmm. i think i'm hitting sqlServer 2014 which is not in the list of supported servers here

@dpsutton also, what you'll want to do tomorrow is find the javadoc for version 3.0 of the driver, since it tells you the other keys supported when you use get-connection

i'll try it

thanks hagmonk. I've got to run now but yall have been super helpful

:thumbsup:

@dpsutton: i just read the jdbc docs and it looks like he tests against both, as far as I know the jtds driver is preferred and recommended over the ms one - I use it all the time and have never had an issue using it to talk to ms sql server.

C

Is there a clojure/core predicate which matches lists, sets, and vectors, but not maps?

In case anyone is into Gradle, I just released the plugin I use to compile Cursive with: https://github.com/cursive-ide/gradle-clojure

@danielcompton: not that I know of

`(every-pred coll? (complement map?))

(every-pred coll? (complement map?))

that’ll do it. Thanks!

@gfredericks If I ever form a hipster band, I’ll call it emacs glitch

@cfleming and only one band member can play at a time, right?

And all the instruments require you to press each string twice with a minimum of three fingers per press.

And each instrument is homemade and not interchangeable, of course

Naturally

I think this idea has legs

(although it should probably be moved to #off-topic at this stage)

one more: the band hasn't improved at all since you saw them first in college, despite everything around them improving a great deal

It has my full support