Fork me on GitHub
#clojure
<
2016-07-27
>
seancorfield04:07:19

@dpsutton: re: simple-query — that’s a pretty old version of java.jdbc (0.4.2)… the API has changed quite a bit since then… You mind find folks running into conflicts if they’re on a more recent version in their code already.

seancorfield04:07:58

(although now I look at your code it looks like you don’t expose any options or anything beyond using java.jdbc/query so you will likely be fine)

dpsutton04:07:01

ah. thanks so much sean

dpsutton04:07:04

i'll bump that

seancorfield04:07:12

Given how query works, you could just pass ~db directly without using with-db-connection — any reason you used the latter?

dpsutton04:07:28

stack overflow lol

dpsutton04:07:44

when first getting into clojure looked up how do sql server

dpsutton04:07:47

so i could sneak it into work

dpsutton04:07:06

it's really worked out well. we've built a whole reporting website for a rewrite that we are building

seancorfield04:07:55

Nice! Always glad to see someone using java.jdbc heavily 🙂

dpsutton04:07:06

are you a dev of that by any chance?

seancorfield04:07:16

Yeah, I’m the primary maintainer.

dpsutton04:07:22

oh wow. well thank you very much

dpsutton04:07:30

its been really straightforward and nice to use

seancorfield04:07:30

I ported it from the old monolithic contrib back in 2011...

dpsutton04:07:41

also super nice to be able to use a lisp in a .NET shop

seancorfield04:07:43

It’s gone through several API rewrites since then

seancorfield04:07:24

We use it with MySQL at World Singles — supporting millions of queries/updates/etc every day.

dpsutton04:07:49

i see you use 0.6.2-alpha2 as the current release. Is it standard for a production release to be an alpha?

seancorfield04:07:58

But I test it with MS SQL Server, with both the MS Type 4 driver and the jtds driver.

dpsutton04:07:01

is there a "stable" version i should leave it on so people feel safe?

seancorfield04:07:14

0.6.1 is the stable version

seancorfield04:07:26

0.6.2 only really has optional clojure.spec support in it

dpsutton04:07:27

I've seen people say to use the MS Type 4 driver but i've got the 3 in there (I think) and it has worked out well

dpsutton04:07:50

this allows me to retrieve from clojars rather than require distribution i believe

dpsutton04:07:57

but i'm not aware of any differenes

seancorfield04:07:03

Well, the few other minor things in 0.6.2-alpha1 compared to 0.6.1 https://github.com/clojure/java.jdbc/blob/master/CHANGES.md

seancorfield04:07:01

The new feature we’re starting to use heavily is :qualifier so we can produce namespace-qualified keys directly from a query...

dpsutton04:07:18

oh i'm interested in that

seancorfield04:07:40

But, yeah, any version I release is likely already in heavy production use at work.

dpsutton04:07:48

i'm not sure how common it is but i built a little function to remove the prefixes of column names

dpsutton04:07:00

at work all of our tables have table names prepended on it

seancorfield04:07:13

Many of the changes since 0.2.x have been driven by what we need at World Singles.

dpsutton04:07:15

happen to have anything like that built in?

seancorfield04:07:41

Well, you could use :identifiers to specify a function to rename columns as they’re turned into keywords.

seancorfield04:07:41

The default is just clojure.string/lower-case but you could pass any function that, given a string, produces a new string (then the library turns it into a keyword).

dpsutton04:07:55

oh ok. i'll look into that.

dpsutton04:07:04

thanks for the heads up

seancorfield04:07:07

So a variant of your remove-prefix would work:

(defn remove-prefix [prefix]
  (fn [s] (clojure.string/replace s (re-pattern (str "^" prefix)) "")))
(java.jdbc/query db-spec ["select * from sometable"]
         {:identifiers (remove-prefix "sometable")})

seancorfield04:07:08

Something like that...

seancorfield04:07:09

(and your first of the result set could be (query ~db [~sql ~@params] {:result-set-fn first}), just FYI)

sgdread04:07:00

@seancorfield: is it possible to do Oracle stored procedures calls with java.jdbc? (don't ask why I want it, it's not my choice :)

dpsutton04:07:20

hey man, i'm working with ms sql server. we all got bills to pay

seancorfield04:07:40

If you can call them via SQL, yes, it should be possible. There’s no special support for stored procs (yet).

dpsutton04:07:43

seeing the syntax it almost removes the necessity for the macros. didn't realize it could be so succinct

sgdread04:07:34

thanks, yep, that 48 exactly I was thinking about

seancorfield04:07:05

@dpsutton: We have a lot of places where we just call (query db-spec ["select * from …" p1 p2]) and that’s it.

dpsutton04:07:40

yeah. our reporting tool is basically just a read only display some tables and do some manipulations

dpsutton04:07:05

didn't know your library had that style syntax. I was thinking it required the jdbc/with-db-connection .... stuff

seancorfield04:07:09

@sgdread: The main obstacle to JDBC-48 is that I just don’t have any exposure to stored procs so I’m not confident about being able to design/test the feature properly 🙂

seancorfield04:07:41

@dpsutton: You only need with-db-connection if you want to get a connection and reuse it across multiple statements, otherwise it’s automatic.

seancorfield04:07:10

And we use a connection pooling library (c3p0) anyway so it’s low overhead to get & close a "connection" on every single query.

dpsutton04:07:10

well sean, you run a nice project there, great documentation, and you are super helpful on slack. thanks so much

seancorfield04:07:24

Always looking for feedback to make it better!

dpsutton04:07:31

i'll keep that in mind

dpsutton04:07:08

but it looks like my principal issue i was working around was nonexistant

dpsutton04:07:18

ps, i found the result that led me into that pattern

dpsutton04:07:44

it might be nice if there was some article, blogpost, whatever that could replace that at the top of google

dpsutton04:07:59

especially since jdbc is so nice to use

sgdread04:07:28

@seancorfield: If you're interested in test cases, I might get some samples of Oracle sprocs with parameters binding. I also know how to deal with them in Java and can share some samples. Oracle has express edition of the database.

seancorfield04:07:45

Is there a Docker image I can easily use? That's how I test Postgres. I can't realistically install Oracle on any of my machines here.

sgdread04:07:05

Eh, need to do a research on docker. Might be a good idea to use them myself.

seancorfield04:07:42

@dpsutton: Hard to compete with SO on Google scores...

dpsutton04:07:53

yeah. shame its old and out of date

dpsutton04:07:04

sean, have you every used vagrant?

dpsutton04:07:18

it makes it really easy to create, provision, use, and then destroy vm's

sgdread04:07:19

vagrant is awesome

sgdread04:07:33

we use it heavily along with ansible

dpsutton04:07:53

you could create a provisioning script that loads up oracle, exposes some ports and then you can hit it from your normal machine

dpsutton04:07:09

then vagrant destroy wipes that thing away, but the provisioning script is still alive

sgdread04:07:31

I actually have some scripts for oracle provisioning in vagrant

dpsutton04:07:45

best thing is that these scripts can be checked in. So you could have a subfolder in jdbc with vagrant files for machines with different db types and a standard database schema/ some data

dpsutton04:07:51

and then run jdbc against them

seancorfield04:07:41

@dpsutton: I kind of skipped Vagrant and went straight to Docker 🙂

dpsutton04:07:08

i started on vagrant but work in .NET so i haven't had the pleasure of docker

dpsutton04:07:25

did vagrant so that all of my school projects still have a version controled environment next to the code

seancorfield04:07:36

With that SO I downvoted the older answer and upvoted the newer one so at least the outdated version will be lower.

dpsutton04:07:37

smalltalk, scheme, whatever, they can all still boot up

seancorfield04:07:29

I'll try to find time tomorrow to edit that answer to make it more up to date and simpler (it's way more complex than it needs to be!).

seancorfield04:07:49

(added to OneNote!)

seancorfield04:07:21

Our stack has a lot of moving parts so Dockerizing it really helps. To run our stack locally you need: MySQL, MongoDB, Discovery (a search engine), Redis, Tomcat, and multiple standalone Clojure apps that all communicate...

dpsutton04:07:54

that sounds super fun

dpsutton04:07:57

being genuine

seancorfield04:07:05

We'll be adding DynamoDB to that mix soon. Docker makes that possible without having developers installing, configuring, and running it all individually.

dpsutton04:07:21

the worst thing to ever hear: "well it works for me"

seancorfield04:07:27

And we're also the crazy company that has a lot of ColdFusion code and we run Clojure inside that 🙂

dpsutton04:07:44

haha. i've never touched it. can't imagine how the two go together

seancorfield05:07:50

I built a bridge library back in 2011 and it's core to our platform...

dpsutton05:07:06

youre a talented guy sean

dpsutton05:07:19

i need these two environments to work together. Sean: I got you

dpsutton05:07:30

jdbc is depcrated. Sean: I got you

seancorfield05:07:07

(modern CFML looks like JS -- the WS "object" is actually a Clojure namespace so WS.data.find_by_keys( name, args, orderBy ) is really just (WS.data/find-by-keys name args orderBy)

dpsutton05:07:59

is there already a java interop so that this isn't as crazy as it seems or did you really build some crazy stuff here?

seancorfield05:07:57

The "magic" code relies on two key things: Clojure's Java API and CFML onMissingMethod() so we can call "non-existent" methods and it re-routes them through a .invoke() call.

dpsutton05:07:33

oh it runs on the jvm. I didn't know anything about it so i thought you were reaching across some really crazy boundaries

seancorfield05:07:59

CFML compiles on-demand to JVM bytecode (just like Clojure). Has done for about 15 years.

seancorfield05:07:25

Prior to that it was interpreted (the interpreter was written in C++ for five years earlier).

dpsutton05:07:27

just really knew the name

dpsutton05:07:38

looking over fw1, that looks like there was some fun stuff

dpsutton05:07:51

and then i'll bet you were really really done with it but still had edge cases

seancorfield05:07:59

We use for some web apps that are pure Clojure

dpsutton05:07:17

i'm looking through your engine library. that seems quite nice

seancorfield05:07:08

It requires discipline to use, but we like what it's making us do... I blogged about it recently...

dpsutton05:07:53

does this require you to parse sql?

dpsutton05:07:03

oh nevermind

dpsutton05:07:12

i thought you were running the sql against the :ram object

seancorfield05:07:14

No, query is based on a protocol and there's an implementation using java.jdbc

seancorfield05:07:36

And an implementation extended to hash maps as well 🙂

dpsutton05:07:53

yeah. for some reason i was thinking that it knew you did updates which hadn't ocmmited yet so it queried the ram instead of the underlying db

dpsutton05:07:15

very nice (re hash maps)

seancorfield05:07:25

The nice thing about the idiom is that it extends to things like message queues, email systems, generator processes etc

seancorfield05:07:20

At World Singles we have Committable data stores that implement all sorts of things... and Queryable data stores for all sorts of stuff too

dpsutton05:07:27

yeah i can imagine data access is your bread and butter

dpsutton05:07:44

i've got to head out. you were super helpful and interesting. thanks

seancorfield05:07:51

Any time. Feel free to DM me if you ever have java.jdbc Qs

Nicolas Boskovic14:07:02

Is there a take-last that returns the individual items I'm looking for wrapped in a vector?

exupero14:07:05

(vec (take-last n coll))

Nicolas Boskovic14:07:34

Say I'm running through a log

Nicolas Boskovic14:07:50

take-last gives me (x line y line z line)

Nicolas Boskovic14:07:59

vec take-last [x line y line z line]

Nicolas Boskovic14:07:09

i'm looking for ([x line] [y line] [z line])

Nicolas Boskovic14:07:31

modding take-last doesnt look like itd be trivial either

robert-stuttaford14:07:58

(map vec (partition 2 (take-last X coll)))

Nicolas Boskovic14:07:27

why partition 2?

robert-stuttaford14:07:57

to break your collection up into subcollections of 2 items each

robert-stuttaford14:07:07

> (map vec (partition 2 (take-last 4 [1 2 3 4 5 6]))) ;; ([3 4] [5 6])

Nicolas Boskovic14:07:55

([line] [line] [line])

Nicolas Boskovic14:07:14

your solution did work though, thanks!

robert-stuttaford14:07:37

> (map vec (partition 1 (take-last 4 [1 2 3 4 5 6]))) ;; ([3] [4] [5] [6]) 😁

robert-stuttaford14:07:01

sequence function composition ftw

robert-stuttaford14:07:16

bit of an addiction, tbh

Nicolas Boskovic14:07:30

I'll turn my workplace into a clojure shop just yet hehehehe

gerrit14:07:38

in the partition 1 case you can simplify it to > (map vector (take-last 4 [1 2 3 4 5 6]))

Chris O’Donnell14:07:52

and if your input is a vector, it would be better to use subvec: (mapv vector (subvec [1 2 3 4 5 6] (- (count [1 2 3 4 5 6]) 4)))

PB14:07:00

Is anyone developing in a docker container using emacs and cider? If so, are you able to jump to fn definitions in a different ns? I keep getting no source locations and https://github.com/clojure-emacs/cider/issues/589 doesnt' seem to address it for me

robert-stuttaford14:07:36

recommend asking in #C0617A8PQ as well, @petr

PB14:07:02

Hah! Good to see you Rob

Nicolas Boskovic15:07:28

any ideas why I wouldn't be able to iterate through that vector?

Nicolas Boskovic15:07:49

I'm trying to run it through a map but nothing's really happening

Nicolas Boskovic15:07:19

(map (fn [ln] (println ln)) line-list)

Nicolas Boskovic15:07:53

is for lazy too? it was also failing with that

petterik15:07:36

I use (into [] <lazy-seq>) a lot

Nicolas Boskovic15:07:56

there should be a flag for non-lazy mapping

cursork15:07:22

Or just use mapv

Nicolas Boskovic15:07:58

and now it seems to get stuck at the last result and stay put forever, boy am I having trouble today

Nicolas Boskovic15:07:53

that's what I get for not changing the text processing function as well, got it.

bit_cmdr16:07:30

Is there a recommended plugin for lein that will generate dockerfiles? I’ve looked at the palletops one, lein-uberimage, but it seems to be abandoned.

jamesnvc16:07:26

alexmiller: I was in here asking about an issue with transit-clj a few days ago; I opened an issue there, but now I’m wondering if that should be on transit-java instead? (https://github.com/cognitect/transit-clj/issues/33)

jamesnvc16:07:43

Since transit-clj appears to pretty much be wrapping transit-java

jamesnvc16:07:06

and I think the issue is in the version of org.msgpack/msgpack that transit-java is using

Alex Miller (Clojure team)16:07:17

I don’t know that it matters much which of those it was filed on

jamesnvc16:07:28

(using signed bytes, I think)

Alex Miller (Clojure team)16:07:42

interesting if it’s a version issue - why do you think so?

jamesnvc16:07:06

the newer version of msgpack seems to be doing the reading of bytes in a difference way

jamesnvc16:07:14

haven’t had a chance to test it yet though

Alex Miller (Clojure team)16:07:45

anything there look relevant?

Alex Miller (Clojure team)16:07:00

actually, maybe that’s not the right lib

jamesnvc16:07:19

oh, it isn’t?

jamesnvc16:07:42

I think the package name changed at some point, from org.msgpack/msgpack to org.msgpack/msgpack-core

Alex Miller (Clojure team)16:07:37

I guess it is - it’s a little confusing but looks like that lib has artifact-id msgpack, so that’s right

Alex Miller (Clojure team)16:07:43

transit-java is using 0.6.10

Alex Miller (Clojure team)16:07:56

looks related - that was fixed in 0.6.11

jamesnvc16:07:05

yeah, exactly that

jamesnvc16:07:27

that is precisely the issue

Alex Miller (Clojure team)16:07:41

I’m a big fan of existing fixes to thorny problems :)

Alex Miller (Clojure team)16:07:50

so that gives you a workaround - exclude the dep and add your own msgpack version

Alex Miller (Clojure team)16:07:01

I’ll prod someone about updating and verifying transit-java

jamesnvc16:07:16

excellent; for now I’ve just been making sure the strings are < 127 or > 255 bytes 😛

eggsyntax19:07:54

I feel like I'm missing something really obvious here:

(defn f [& {:keys [a b c] :as args}]
  (println a b c))

(defn g [& {:keys [a b c] :as args}]
  ;; do some stuff & then pass args on to f
  (f args))
(f args) throws an error. I can do it as (apply f (flatten (seq {:a 2 :b 3 :c 4}))), but that's totally ridiculous. What idiom am I forgetting here? Or of course I can do it as (f :a a :b b :c c) but that's manual and annoying.

exupero19:07:44

Remove the & from the arglist if you want it to take a single argument.

eggsyntax19:07:35

Hmm. Less convenient for anyone calling it directly, though.

eggsyntax19:07:51

(as a subjective matter of taste)

dpsutton19:07:15

maybe try a true multi-arity setup rather than the optional parameters?

eggsyntax19:07:04

@dpsutton: I don't love that option, because I want it to be easy for people to omit any subset of params.

hans19:07:09

optional parameters and destructuring rest arguments have become quite unpopular for their lack of composability.

eggsyntax19:07:37

I still like 'em 🙂. But definitely awkward in this case.

dpsutton19:07:44

wait, is args optional or not?

dpsutton19:07:59

if it is, i don't see why you don't want a 0-arity version

dpsutton19:07:09

if its not optional, remove the & as mentioned earlier

hans19:07:40

eggsyntax: maybe you should use common lisp instead, for it's excessive lambda list madness. there you get all you want and more in argument list specification 🙂

eggsyntax19:07:04

@dpsutton: Not sure I understand what you're saying. Desired calling convention is like (g :a 2 :c 3) (or any variant thereof).

eggsyntax19:07:18

@hans: fair enough, that would certainly be a solution 😉

dpsutton19:07:27

oh i'm with you now

dpsutton19:07:43

yeah. common lisp has a nice way of setting defaults i believe

hans19:07:54

eggsyntax: if you stick with clojure, though, i can really suggest that you forget about believing that it has keyword arguments

eggsyntax19:07:55

I'm just used to clj destructuring being so rich that it surprised me there isn't an obvious way to do it. But sounds like there's not.

dpsutton19:07:10

i was misreading destructuring keys from an object, not from a rest type

hans19:07:18

dpsutton: not only that, but you can also determine whether a certain keyword has been specified or defaulted.

dpsutton19:07:34

yeah. are you against making a map and merging with a notion of default?

hans19:07:40

dpsutton: and you can rename the variable that the argument is bound to

eggsyntax19:07:49

I do miss Python's handling of optional args; it's one of the few things I miss in Clojure.

hans19:07:59

really, common lisp lambda lists are something to drool about.

hans19:07:02

sorry for OT

dpsutton19:07:03

i'm in .Net land at work and optional stuff is really nice as well

eggsyntax19:07:03

@dpsutton: nah, that'd work too 🙂

dpsutton19:07:20

but i agree with you now that it's not as nice syntax

exupero19:07:23

@eggsyntax: You can provide default values in the destructured arg with :or.

hans19:07:39

exupero: which is workable, but fugly

eggsyntax19:07:08

(note on the Python version: it was great for the common case, but had a deeply ugly pitfall: defaults were set at function definition time, which meant that if the default value was an array or other mutable object, changes to it would persist across function calls.)

richiardiandrea20:07:37

Which is the most reliable SMTP server one can use? I found https://github.com/whilo/bote which is kind of interesting, it uses SubEtha SMTP

hiredman21:07:41

why not use subethasmtp directly?

hiredman21:07:51

bote doesn't seem to give you much of anything

hiredman21:07:10

subethasmtp is good

richiardiandrea21:07:35

yeah @hiredman I think you are kind of right

richiardiandrea21:07:03

I found an odd behavior, maybe it is expected, but just wanted a confirmation if anyone know

richiardiandrea21:07:22

(every? true? nil) ;=>> true (every? true? [nil]) ;=>> false

hiredman21:07:15

Clojure 1.9.0-master-SNAPSHOT
user=> (every? true? [nil])
false
user=>

richiardiandrea21:07:23

the first line is odd and I just found out a test was passing but it shouldn't have

richiardiandrea21:07:29

sorry I corrected above

hiredman21:07:44

every? is always true on an empty collection

richiardiandrea21:07:38

oh right...it is the old story of an AND when no items are tested? There is an example actually here: http://clojuredocs.org/clojure.core/every_q

richiardiandrea21:07:53

yeah I was kind of forgetting that

richiardiandrea21:07:15

it is a 101/interview question 😄

dg21:07:39

I hope I'm not asked questions like that in interviews

richiardiandrea21:07:42

well, I have made my gaffe of the year, I can now retire 😄

eggsyntax21:07:50

Sometimes I find it helpful to mentally translate to: “there is no item in coll which does not satisfy pred."

richiardiandrea21:07:11

I actually had an interview yesterday and one of the questions was why in Java (Integer.valueOf(-128) == Integer.valueOf(-128)) is true (I answered that it should be false)

richiardiandrea21:07:26

but ok that's not a logic question

richiardiandrea21:07:57

it is a side effecting Java quirk of caching objects for number within some range

dpsutton22:07:18

ah that's because integers are objects and the common ones are created at start up but when you ask for ones outside of a small core range they are newed so to speak? And the == test here is using reference equality?

martinklepsch22:07:26

is there a common term for things that manage resources that can have multiple users and dispose them when there's no user left?

artiavis22:07:51

Yeah Python has that behavior for boxed integers as well. All the boxed integers are singletons, and the ones in a small range around 0 never get GC'd.

artiavis22:07:08

Interning, I think?

Alex Miller (Clojure team)23:07:49

The Java cached primitives are never disposed

Alex Miller (Clojure team)23:07:07

It's actually mandated in the spec

lprefontaine23:07:18

Can someone explain to me if such a question in an interview has any value (no pun intended) ?

austb23:07:02

@lprefontaine: creating quality tech interview questions is a difficult thing because most are subjective and on a job you could look it up. But I agree that it seems particularly and purposefully obscure to try to have the (maybe stressed) interviewee identify that you’re comparing two objects using == and then also knowing about Java caching certain numbers

richiardiandrea23:07:22

@alexmiller: yes it is in the specs, I did not know that

richiardiandrea23:07:52

didn't want to open a flame as I guess we all have our idea on what to ask in interviews, the interviewers were actually superfriendly and wanted to go in deep but always smiling and knowing that these things are kind of...well...imho worthless

lprefontaine23:07:24

In my world I deliver value. The fact that an implementation has such quirks is irrelevant to what I can deliver. The industry seems to be in such a disarray to select candidates that I am not surprised of its lack of throughput if this is the kind of discriminant factor it's using to select 'the best candidate's. I would have left the interview at the first question of this kind. Everyone should. This world is sick.

richiardiandrea23:07:39

at least now they know that I did not read the Java spec, for them is worth knowing that

richiardiandrea23:07:54

btw I got an offer

richiardiandrea23:07:02

so the interview went well

richiardiandrea23:07:28

because it is clear maybe that you can go deep, but also clear that you cannot know everything

richiardiandrea23:07:35

which is healthy

austb23:07:58

Congratulations on that. And yes, I feel like it’s probably also a benefit to know someone didn’t just spend their free time reading the Java spec for no particular reason

lprefontaine23:07:27

I would rather read some byte code on a sheet of toilet paper 😬

richiardiandrea23:07:07

^ many lol (it is a Java job so now I dunno what to do, other flame open :D)

lprefontaine23:07:27

have a nice start on your new job, I'll get back to my clojure/jdbc/oracle/unfamiliar schema problem...