Fork me on GitHub
#clojure
<
2016-08-17
>
verma01:08:59

thanks @seancorfield for weighing in, at this point after looking at some of the options, I think I might go with HugSQL (or YesQL) since these give me the right blend of magic and flexibility

verma01:08:47

Some of the more experienced DBAs would be helping me so I guess it’d be nice to give them the flexibility they need with SQL queries

verma01:08:51

I am going to watch the HoneySQL video and see where it takes me

gja02:08:21

@verma: strongly recommend honeySql. As your domain evolves, your queries will get more complex. You’ll keep having this scenario where “I want the same as this other query, except… x” where x in (add new select field, add another where clause, 100 other minor tweaks). In this scenario, having your SQL query represented as a clojure map is really helpful, as it’s programmable

dpsutton03:08:16

i always take the opposite view. especially if you have dba's, if you are writing in some DSL and not SQL you are alienating people who can help you. It also means you can't profile, test, and develop the queries against the standard clients (SSMS for us with MS Sql Server), and then you run up against aggregates not working correctly, perhaps unicode/ascii casts that get expensive, etc. For all but the simplest tasks i always prefer to write my own queries

seancorfield05:08:14

@verma: Yes, I think if you're going to be working with DBAs then you want something that will let you work primarily with SQL directly.

seancorfield05:08:38

I recently added an :explain? option to java.jdbc so you can get a report from (most) DB of how a query will work for a DBA to review. Something I've been asked for several times and just haven't figured out a good way to handle it is the ability to retrieve or print the actual SQL sent to the DB (which is useful when dynamically building queries such as with HoneySQL).

seancorfield05:08:00

We've found :explain? "explain extended" very useful with MySQL...

seancorfield05:08:35

(the default :explain? true is equivalent to :explain? "explain")

robert-stuttaford05:08:36

@grzm: are you grzm on twitter?

Pablo Fernandez07:08:31

alexmiller: this is not what I’m using, but what my libraries that I hope other people will use will depend on.

gerrit07:08:08

@pupeno: I think adding :scope "provided" to your clojure & clojurescript dependencies makes your library users get the version they want

Pablo Fernandez07:08:25

gerrit: ahhh… 🙂 thanks.

vinnyataide08:08:08

hello, what would be the disadvantage of targeting node with cljs instead of using Java with clojure?

vinnyataide08:08:40

I mean node is pretty fast and lightweight

vinnyataide08:08:35

and what would be a replacement for tomcat that would mimick the one thread scerario of node

val_waeselynck08:08:13

@vinnyataide: smaller server-side ecosystem for ClojureScript than Clojure; slightly restricted Clojure language features (macros are less dynamic etc); but most importantly, the differences between the JVM and NodeJs as platforms

vinnyataide08:08:30

but not equivalent to clojure

gnejs08:08:33

@vinnyataide: if you’ve got a library that can provide thread-safe datastructures (clj) why not take advantage of a multi-threaded server? What would a single-threaded http-server give you?

val_waeselynck08:08:47

@vinnyataide: in particular, sometimes you do like to use blocking IO

gnejs08:08:50

(but maybe I didn’t understand the point :))

vinnyataide08:08:36

nah, I like my code horizontal 😜

vinnyataide08:08:44

I mean with callbacks and such, but I see the importance of branched interactive scripts

vinnyataide08:08:55

and other things

vinnyataide08:08:02

@gnejs: the speed of node is because of the single thread service afaik

genRaiy09:08:38

@vinnyataide: performance comes from async IO rather than single threading per se. node.js scales well compared to old-skool java servlets because it does not use a thread per request and thus uses less RAM. However, having async IO plus MT means that you can use more of the hardware without going into cluster mode in node.js

gnejs09:08:06

The biggest problem with enterprisey java servers are that they are not written with async/reactive in mind. Neither are the older applications running on them. Netty/Jetty/Undertow are implemented differently. Vrooooom. And can utilize multiple CPUs from one process.

vinnyataide09:08:54

that's a really nice answer

vinnyataide09:08:59

I got a friend that deployed a service in node and told me that It went 30% less resources than other alike that was in java

val_waeselynck09:08:14

@vinnyataide: there's no doubt here, NodeJs generally has a smaller footprint

val_waeselynck09:08:29

especially for IO-heavy loads

vinnyataide09:08:08

very nice lib

vinnyataide09:08:28

so the architecture that I'm building that is message based is comming through is datomic as database, yada server with om next, and front end with om next with secretary for routing. Is that right?

vinnyataide09:08:56

I saw apache kafka as a message pub service but Idk how it would fit since I've never used it

val_waeselynck09:08:43

@vinnyataide: sounds good! Since you're interested in making non-blocking apps, keep in mind that Datomic will prevent you from going full-async for reads, because loading segments from storage to local cache is synchronous; however, thanks to pervasive caching there won't be that much IO at all and the latency should be low (writes are asynchronous on the other hand).

genRaiy09:08:25

@vinnyataide: IMHO Kafka is better for event streams rather than messaging (i.e. recording events rather than using messages to put work on a queue)

vinnyataide10:08:44

@raymcdermott hmm that's good to know

vinnyataide10:08:00

@val_waeselynck: yeah that's okay for me

genRaiy10:08:17

@vinnyataide: AMQP is typically better for messaging (with Kafka you have to manage consumers into different partitions so you’re starting with a hack)

val_waeselynck10:08:12

@vinnyataide: HornetQ's pretty good too, and another reason to use the JVM 🙂

vinnyataide10:08:19

well the whole problem that I'm trying to solve is live reports for my clients

vinnyataide10:08:00

cool, so many options, I'm kinda new to the java ecosystem

vinnyataide10:08:44

I'm not going yet into these, but I'll take a look at it later, the writes are my focus right now, the queries and sync will be focused later, I know that with om next I can do that with complete decoupling

plexus10:08:14

FYI you can now link to individual messages on the Clojurians slack log, e.g. https://clojurians-log.clojureverse.org/clojure/2015-06-04.html#inst-2015-06-04T13:03:37.002003Z

plexus10:08:24

with props to @grzm

plexus10:08:59

(still regenerating all the static html so give it a minute)

vinnyataide10:08:12

looks like I'll be choosing sente as a lib only solution for the reports

mikethompson12:08:40

In clojurescript I can write this predicate:

(defn deref? [x]
  (satisfies? IDeref x))
What is the clojure equivalent ?

joost-diepenmaat12:08:13

(instance? clojure.lang.IDeref x)

mikethompson12:08:45

Ohh, I see. Thanks. That's confusing: cljs.core/IDeref in clojurescript and clojure.lang.IDeref in clojure.

joost-diepenmaat12:08:09

IIRC in clojure/jvm IDeref the java Interface was written before protocols were introduced to the language. Clojurescript tends to make more use of protocols.

mikethompson12:08:56

I had read that somewhere. But this makes it more concrete, thanks.

joost-diepenmaat12:08:26

probably backwards compatibility is holding clojure/jvm back a bit in that respect

Alex Miller (Clojure team)13:08:50

it’s more then that - would imply re-working a significant portion of the code base

verma13:08:28

gja: what you’re saying does make sense, but I am in no way an SQL expert, so would rely on people who do this for a living, I’d have to decompose our DBA’s queries and then recompose them using clojure data structures while considering re-usability and composition, also the DBAs may send me updates to queries which I’d have to rework again, I will take a look at HoneySQL and see if it changes my opinion 😕

verma13:08:15

seancorfield: that makes sense, I will take a look at it, thanks :thumbsup:

grzm13:08:49

@plexus: thanks for pulling this 🙂

gja14:08:34

@verma @dpsutton you can always translate to sql, and have DBAs profile it. HoneySQL does do a very good job of correctly casting things, much better than I do myself, personally. But if the DBAs are giving you the raw SQL, and you are sure that you’ll never make any changes, then i’d pick HugSQL.

dpsutton14:08:15

we had a complicated-ish query that had a few group-by's and an aggregate, maybe max or sum, over microsoft sql server. honey lacked that at the time, if it doesn't still lack that

verma14:08:37

thanks for weighing in gja :thumbsup:

dpsutton14:08:33

oh just found out they did have the aggregates, but they aren't called aggregates. You use % to specify functions

ddellacosta16:08:49

@dpsutton: even if it lacks that kind of thing, it’s really easy to extend

ddellacosta16:08:50

HoneySQL is a good choice when you want to do some basic composition of queries programmatically, and/or do a lot of substitution of fields or variables

ddellacosta16:08:27

per gja’s point, definitely makes sense to use something like HugSQL if you are basically going to drop some queries in and use them without much variation—in that case HoneySQL is too much work I think

spieden16:08:47

i’m using honeysql for building up queries from common patterns this week — working quite nicely

spieden16:08:04

@dpsutton: for more complex function calls there is honeysql.core/call

lucasbradstreet17:08:33

Hope you guys don’t mind us spamming #C03S1KBA2 a little. #C051WKSP3 just closed a seed round to bring a managed, clojure first, data solution to the masses http://www.onyxplatform.org/jekyll/update/2016/08/17/Funding.html

mattly17:08:51

I'm using honeysql as the backbone for a very complex query editor

mattly17:08:09

based on user input I have to rope in one to twelve joins, columns, etc

mattly17:08:12

it works wonderfully

sdegutis17:08:37

Is there something that lets me call a Clojure server from a ClojureScript front-end, as if it was in the same process, but only assuming EDN arguments and return values? Some kind of RPC-like framework?

robert-stuttaford17:08:51

@sdegutis: cljs-http is perfectly fine for this

robert-stuttaford17:08:25

why do you need it to feel like RPC? past experience has taught me that trying to hide HTTP usually ends badly 🙂

sdegutis17:08:46

@robert-stuttaford: Assuming I'm making a ClojureScript SPA, then the backend doesn't really need to have "routes" per se, since the routes are all handled on the front-end, which does all the user-facing stuff. This leaves the back-end just being a dumb business-logic server that has functions that return values.

sdegutis17:08:04

Routes are only necessary in the context of a browser, and when that's handled via JS/Cljs, the JS/Cljs doesn't have any need to keep using the "route" metaphor to communicate with the backend.

spieden17:08:17

my web apps mostly use just two backend rest resources for everything: one for datomic queries and one for commands

spieden17:08:33

then i add other stuff as needed that doesn’t fall into those buckets

sdegutis17:08:03

Right but REST is only necessary when interoperating with other services. When your back-end and front-end are tightly coupled, they can just use EDN, and at that point they don't even need routes.

spieden17:08:12

so yeah, no transparent function calls but commands are data that end up being evaluated by function calls

spieden17:08:04

yeah i guess i could reduce my two endpoints to one but shrug

spieden17:08:29

i use transit instead of edn

spieden17:08:36

which is probably a premature optimization

sdegutis17:08:30

Maybe not. Depends on how much data you're passing. EDN can be lengthy.

spieden17:08:30

calling them rest resources is probably inaccurate too i suppose — they’re just HTTP resources with a particular contract

sdegutis17:08:46

Meh, nobody really means the same thing when they say REST anyway 😉

spieden17:08:53

yeah, i heard that transit can leverage the native JSON parsing in the browser too

robert-stuttaford18:08:08

@sdegutis: cljs-http + an endpoint that runs all POSTed edn through a defmulti has worked just fine for us

robert-stuttaford18:08:56

the defmulti affords us all the organisation we need, because we can implement defmethods anywhere. that plus a sufficiently sophisticated dispatch value - e.g. [entity-type action], and you're golden

sdegutis18:08:32

And then a third argument just takes optional data?

robert-stuttaford18:08:34

client simply invokes the same fn for all calls, passing different data, and uses async to get results (because cljs-http uses async)

robert-stuttaford18:08:42

well, you're passing a map, so all the data is in the map

sdegutis18:08:49

Right, right.

sdegutis18:08:05

Are you using Datomic btw? I don't often see the term "Entity" used outside it.

robert-stuttaford18:08:35

(defmulti api (juxt :type :action)) (defmethod api [:user :add] [{:keys [email password]}] ...)

sdegutis18:08:54

I like this plan a lot.

robert-stuttaford18:08:16

(POST "/api" request (-> request :body util/read-edn api util/edn-response))

robert-stuttaford18:08:44

of course, you probably want to pass request along as well so that you can deal with session etc

robert-stuttaford18:08:49

and you probably want to use nicely factored transit middleware instead of this lazy threading of edn functions - but the thread shows what should happen

robert-stuttaford18:08:29

it's simple, immediately obvious what's going on, and it scales quite well!

spieden18:08:48

yeah same idea here — commands are handled by a multimethod

spieden18:08:14

i still like having liberator for error handling, ser/deser, etc.

robert-stuttaford18:08:57

yeah. makes sense to have that stuff in middleware

dpsutton19:08:22

is there a mapc version of map explicitly for side effects? Or is it just best to (doall (map ... ) )

pesterhazy19:08:32

there's run!

exupero19:08:57

@dpsutton: If all you need is side effects there’s doseq.

dpsutton19:08:25

ah yes. thanks

spei19:08:02

@dpsutton

(dorun (map ... ) )

dpsutton19:08:50

is that preferable to doall?

jr19:08:32

mapv is eager if that’s what you want

chris19:08:46

doall holds the whole seq in memory

chris19:08:54

and returns the seq

dpsutton19:08:56

ah, yeah i guess there's no such thing as a lazy vector

dpsutton19:08:08

thanks chris. good to know

timgilbert20:08:35

Say, anyone know of a reasonably fast command-line tool for pretty-printing edn? And ideally terminal-coloring it as well?

timgilbert20:08:07

I was hoping to cook up something fast in planck, but it doesn't seem to implement clojure.edn/read

timgilbert20:08:52

Oh, perfect, thanks @alexmiller!

camdenclark21:08:03

Is ring + compojure the best stack for creating web app backends right now?

danielstockton21:08:47

I'm trying to implement a custom data_reader, it appears to be called twice. It does (swap! atom dec) and i see the number decreasing by 2. Can anyone suggest why?

donaldball21:08:05

Lots of people like ring+compojure. Personally, I think any routing library that can’t provide the inverse function is incomplete. I’d reach for bidi if I were using ring nowadays, I think, and would take a long hard look at liberator.

donaldball21:08:53

Pedestal has a lot to offer, that’s what I’m using now fwiw

danielstockton21:08:16

Solved it myself, apparently I have to quote the form to prevent it being evaluated twice

camdenclark21:08:36

@donaldball: pedestal seems great! thanks for the heads up, I’ll probably work with that

spieden21:08:38

getting mileage with bidi + liberator over here

dpsutton21:08:24

@donaldball: what do you mean by the inverse function?

spieden21:08:24

dpsutton: e.g. {:action “foo”} -> “/action/foo"

spieden21:08:28

so you can take the data representation of a location and produce something the client can make a URL with

carocad22:08:17

hey guys, does anybody know why does the nrepl outputs exceptions filenames as 'form-init<random-number>' even if the filename, line and column are set on an eval call?

kenny22:08:45

In 1.9.0-alpha10, is map destructuring not going to be backwards compatible? For example, in clojure 1.9.0-alpha10:

(let [{:keys [foo/bar]
       :or   {foo/bar [1]}} {}]
  bar)
=> nil
And in 1.8.0:
(let [{:keys [foo/bar]
       :or   {foo/bar [1]}} {}]
  bar)
=> [1]

jstokes22:08:45

anyone know why this is?

(/ 100 (get {} :foo 0.0)) ;; => throws ArithmeticException
(/ 100 0.0) ;; => Infinity

dpsutton22:08:21

ah i'm sorry i misread

jstokes22:08:30

why is the behavior different for the missing key entry with default? or when the key is in the map, even: (/ 100 (get {:foo 0.0} :foo)) => throws exception

dpsutton22:08:25

(class (get {} :foo 0.0)) java.lang.Doubl

dpsutton22:08:28

super strange

dpsutton22:08:52

is there some type casting optimization that can't be done here?

dpsutton22:08:11

in the second, analyzing the types clojure can see that 100 is an int but 0.0 is a double so do double division?

jstokes22:08:19

hm, (/ 100 ^double (get {:foo 0.0} :foo)) ;; => Infinity

jstokes22:08:23

i think you’re onto something

dpsutton22:08:44

i did the same without the foo tag in the map and it produced infinity. seems like the runtime has already commited to integer division

dpsutton22:08:55

does that mean this isn't a truly eager evaluation method?