Fork me on GitHub
#clojure
<
2018-01-08
>
dpsutton02:01:52

bronsa mentioned in his decompiler talk from baltimore that he wrote a backend for core.match that didn't use exception based backtracking. I'm not seeing that on github. was that patch accepted?

alexmiller02:01:18

I don’t think it got looked at before 1.9

dpsutton02:01:33

quite busy at the time i guess 🙂

dpsutton02:01:54

what an amazing talk that was

dpsutton02:01:05

and he wasn't kidding about the steps in decompilation. this is basically right from a slide in the talk

(defn class->source [classfile-or-classname bc-for]
  (-> classfile-or-classname
      (bc/analyze-class)
      (ast/bc->ast {:bc-for bc-for})
      (sa/ast->sugared-ast)
      (src/ast->clj)
      (cmp/macrocompact)
      (pp/pprint)))

alexmiller02:01:51

oh, I’m thinking of something different. I’m not sure that backend was ever submitted to core.match

dpsutton02:01:09

yeah i looked on jira and github and didn't see anything

alexmiller02:01:22

well @bronsa is on here, you should just ask him :)

dpsutton02:01:52

i don't like pinging people when they aren't active at the moment. really should have been off topic. just a great talk

tjtolton03:01:22

is it possible to "monkey patch" a symbol in clojure? So that whenever someone calls clojure.tools.logging/info for example, it calls my function instead? When responding, bear the following 2 things in mind: 1) hook does not work on macros, which is the reason I'm asking 2) Yes, I know this is ugly, and shameful. I'm doing this for my company, we've got a deadline. I'll do it right when I have time to sleep, which is when I'm dead.

noisesmith03:01:08

the only way to replace a macro is before the code that used the macro is compiled

noisesmith03:01:41

if hook can't do that to a macro, then you can replace the macro yourself, but either way it won't work unless you also recompile all the code using the macro

noisesmith03:01:23

the easy choice here is to fork clojure.tools.logging, give it your own group id, and use that instead of the original

tjtolton03:01:32

the easy choice here he says 😂

noisesmith03:01:35

it's less complex than the other options

tjtolton03:01:43

I wish you weren't correct.

noisesmith03:01:17

slightly trickier, you can edit the code of clojrue.tools.logging and deploy to your local cache

noisesmith03:01:40

that way you avoid some dep management complexity in making suer everyone uses your version

noisesmith03:01:57

putting it in your local cache is a question of running lien install - that's it, that's all it takes

tjtolton03:01:32

lol. Haven't worked with devops engineers, have you?

noisesmith03:01:54

OK, so you can't use something you jarred locally

tjtolton03:01:54

would be faster for me to refactor every use of the namespace in the codebase 😂

tjtolton03:01:25

yeah, I'm bummed, seems like there's no way around a deep solution

noisesmith03:01:42

well, using your own version of tools.logging is just a question of adding some exclusions to your deps and then deploying / depending on your version

tjtolton03:01:39

yeah, refactoring all the requires, etc

noisesmith03:01:48

no, you don't need that

noisesmith03:01:59

all you need is to change the dep in your project file

noisesmith03:01:36

your fork of tools.logging would define all the same namespaces

tjtolton03:01:43

ahh, I see, thus the dep management. right, right.

tjtolton03:01:05

hmm. yeah, I might be able to make that work :thinking_face:

noisesmith03:01:10

or, the other route is to use the normal dep, and have a namespace the does the following: switch to the core.logging namespace, redefine the macro, cause all other namespaces to be reloaded from source

tjtolton03:01:53

how would one force a reload like that?

noisesmith03:01:26

(doseq [name-space (keys (all-ns))] (require name-space :reload))

noisesmith03:01:52

if that doesn't suffice you can look at what clojure.tools.namespace does

noisesmith03:01:34

you would have to fix that to exclude the logging ns you are replacing of course, heh

tjtolton03:01:45

hahahaha, I was just thinking that

noisesmith03:01:51

but given those ingredients,the rest is just an exercise

noisesmith03:01:14

the downside is everything gets compiled twice in order to make sure your replacement macro is working

noisesmith03:01:19

which means startup is slower

dpsutton03:01:28

probably can't help with the problem but can you describe the issue that you're running into?

tjtolton03:01:51

situation is pretty simple. I want to log in json instead of edn

tjtolton03:01:00

whole codebase currently uses clojure.tools.logging. clojure.tools.logging is a chain of macros that cant be hooked. By the time I can hook, all the logging arguments have just been concatenated into a big unstructured string.

tjtolton03:01:06

I would have to regex it apart to figure out what the data map is and what the message is, parse it back into an edn map, then dump it into a json string.

tjtolton03:01:08

if I could just hook it at the very beginning, I could cheshire that stuff into place before all the damage was done.

tjtolton03:01:24

So @noisesmith was helping me figure out how to "hook a macro"

tjtolton03:01:09

I think startup time hit is acceptable. :thinking_face:

tjtolton03:01:33

thanks a lot, noise. This is some really helpful advise, and I sure needed it

dpsutton04:01:00

The logging implementation can be explicitly provided by using
            binding or alter-var-root to change the value of *logger-factory* to
            another implementation of clojure.tools.logging.impl/LoggerFactory
can you make your own with cheshire serializing the edn?

tjtolton04:01:02

well, the logger factory that I implemented would still be passed an unstructured string that typically looks something like this: "A thing happened!: {:some \"info\"}"

tjtolton04:01:21

assuming the function was called as: (clojure.tools.logging/info "A thing happened!:" {:some "info"})

tjtolton04:01:02

so you see it's kind of already too late by the point it hits the logger

tjtolton04:01:14

Id have to run some weird regexes to split the data from the message, parse the data into memory from the string, assoc in the message as a keyval, then dump it to json using cheshire

tjtolton04:01:32

unless there's something I'm missing, which please god tell me there is

tjtolton04:01:50

honestly its just madness to me that it was implemented the way it was. clojure.tools.logging, that is. There must be some kind of crazy performance benefit to using macros over functions that I'm unaware of.

dpsutton04:01:32

oof. yeah i'm seeing that. it looks like its made for logging unstructured data since it's just calling print-str on the two args

dpsutton04:01:12

are all calls to the info logging fully qualified?

dpsutton04:01:35

you could perhaps make your own info macro that calls the logger in a slightly different way

tjtolton04:01:59

yeah, proxing it and refactoring all the requires is an option

dpsutton04:01:05

(log* logger# ~level x# (print-str [email protected])) would do it for you. the branch taken when its a throwable

dpsutton04:01:39

but it has to be a macro so it doesn't eval args when the level is lower than the logging theshold. or require everything to be wrapped in a thunk

tjtolton04:01:57

ahhhh, is that why? whoa. I had no idea that optimization was even in place. so I could do a (logging/debug "hello" :wait (Thread/sleep (* 100000 100000 100000))) and nothing would happen if my log level was set to error?

tjtolton04:01:27

what a strange thing that is

dpsutton04:01:48

we bult a logger in C# at my last job that would collapse a bunch of contexts to give a snapshot of the world at a logging call. expensive if thrown away all the time

tjtolton04:01:48

Is that useful?

dpsutton04:01:07

think

using (new LoggingContext(a = 3, b = 4)
using (new LoggingContext(c = 5 )
Log("blah blah");
would include the variables you declared to push into the "pancake" we called it

dpsutton04:01:14

very useful to give extra context to an error. it would put invoice number, customer number, rule, etc when there was an error. it was a stack trace and a snapshot of important things

tjtolton04:01:37

yeah.. thats a fascinating use case for macros. well, I mean I guess short-circuiting is like the oldest macro use in the book

tjtolton04:01:42

preventing the evaluation of arguments completely. I mean I guess yeah of course that's a thing...

dpsutton04:01:20

(defmacro info [level x & more] (logp ~level nil (assoc (first more) :message ~x)) might work for you

tjtolton04:01:10

thanks, mr. sutton! I suppose it will come down to something like that in the end

dpsutton04:01:29

(defn foo
  "I don't do a whole lot."
  [x]
  (log/info :info "hi" {:stuff 4})
  (hijacked/info :info "hi" {:stuff 6})
  (println x "Hello, World!"))
yields
(foo 4)
Jan 07, 2018 10:38:14 PM clojure.tools.logging$eval429$fn__432 invoke
INFO: :info hi {:stuff 4}
Jan 07, 2018 10:38:14 PM clojure.tools.logging$eval429$fn__432 invoke
INFO:  {:stuff 6, :message hi}
4 Hello, World!
nil

dpsutton04:01:56

(ns fact.logging
  (:require [clojure.tools.logging :as log]))

(defmacro info [level x & more]
  `(log/logp ~level "" (assoc [email protected] :message ~x)))
@tjtolton

tjtolton04:01:55

yeah, gotcha. Trick is, of course, that I'd have to drop that as the first line of every namespace that uses logging forever after amen. And I also have to rely on the argument being a map. If it is, in fact, an unevaluated function or something (similar to the use case you described), then it's gonna break.

dpsutton04:01:35

you were saying you could turn it into json. so i assumed there was some kind of convention

dpsutton04:01:56

but yeah its fragile for sure

tjtolton04:01:35

yeah, there's probably some loose use of convention. Im sure that at least half of the uses are passing in individual keyvals too, rather than the whole map

tjtolton04:01:38

not a hard lift

tjtolton04:01:49

just little extras here and there to consider.

dpsutton06:01:09

ah. the backend for core.match is defined in decompiler itself.

bronsa13:01:11

@dpsutton glad you liked the talk :) re: the core.match frontend, ATM it’s just defined for tools.decompiler, the reason why I’m not making a patch for core.match is twofold: - that new frontend is only defined for the subset of core.match that tools.decompiler uses, it could be made general but I haven’t had time to do so yet - for “normal” match statements, the current frontend clearly wins — it’s only for very complex, wide and nersted match statements that the one tools.decompiler uses starts being faster

madstap16:01:19

Is the talk online? If so, could you post the link? I couldn't find it after a couple of searches.

mbjarland14:01:41

< question moved to #leiningen >

markbastian16:01:10

Got an interesting issue I was wondering if anyone in the community might be able to help me with. I'm trying to use clojure.jdbc to call a custom function in H2 with a user-defined variable. It doesn't work. Here's the code:

(for [q ["CALL ECHO('1')"                                   ;Works
         ["CALL ECHO(?1)" "2" ]                             ;Works
         "CALL ECHO(@X)"]]                                  ;Fails
  (let [db-spec {:classname "org.h2.Driver" :subprotocol "h2:mem" :subname "demo;" :user "sa" :password ""}
        alias "CREATE ALIAS ECHO AS $$ String echo (String s) { return s; } $$"
        ;Need to keep a local connection handle due to how I spec the db.
        c (j/get-connection db-spec)]
    (try
      ;Create a local user variable
      (j/db-do-commands db-spec [alias "SET @X='3'"])
      ;Call my custom function. Note that using the user variable fails. Why?
      (j/query db-spec q)
      (catch Throwable t {:error {:message (.getMessage t)}})
      (finally (.close c)))))
Result:
=> (({:public.echo('1') "1"}) ({:public.echo(?1) "2"}) ({:public.echo(@x) nil}))
Any ideas why? TIA!

dpsutton16:01:05

is this a new 1.9 feature? (let [{:a/keys [b c]} {:a/b 1 :a/b 2}] .... being able to prefix ;keys with the all of the keys will have?

Jakub Holý (HolyJak)16:01:37

Is there something like Project Euler but focused on learning Clojure syntax, code functions and core language? I love P.E. feature of comparing my solution with the top 10 Clojurians however I spend too much time with mathemstics and thus too little eith the lang. Thanks!

jholkeboer16:01:59

(resolve (symbol "+")) returns #'clojure.core/+, but (resolve (symbol "System/currentTimeMillis")) returns nil. is it possible to parse java functions like this from a string?

noisesmith17:01:44

along with the other issues mentioned, a java method is not first class, while clojure functions are (clojure functions are objects with invoke methods which is why they are first class)

bronsa16:01:27

you’ll have to use reflection

bronsa16:01:55

and know the type signature

hiredman17:01:47

user=> (macroexpand '(System/currentTimeMillis))
(. System currentTimeMillis)
user=> 

jholkeboer17:01:46

didnt know that the / is basically a macro. interesting

bronsa17:01:39

only for static field access/static methods

bronsa17:01:44

not for namespaces

bronsa17:01:03

even .foo is a macro that expands to .

bronsa17:01:22

user=> (macroexpand-1 '(.toString 1))
(. 1 toString)

Quan Nguyen17:01:18

That's news to me, thanks for sharing!

joshkh18:01:50

i'm stumped regarding a problem involving multimethods. i have ProjectA with a defined multimethod and ProjectB which extends the multimethod from ProjectA. ProjectA requires ProjectB thus adding extra functionality to ProjectA. how can i go about developing and compiling ProjectB when its defmethods refer to the defmulti in ProjectA? to avoid the circular dependency i could create a third project with just the multimethod, but that seems a little crazy.

hiredman18:01:17

this is a common problem people face with namespaces within a single project

hiredman18:01:00

basically your definition of an interface (defmulti) should not depend on an implementation of that interface (defmethod)

joshkh18:01:27

is there common technique for getting around that?

hiredman18:01:36

maven, if I recall, is perfectly fine with circular dependencies between artifacts if I recall

joshkh18:01:19

that's true - but it produces a warning and for some reason that bothers me, ha.

hiredman18:01:24

I dunno, I just avoid having the definition of an interface depend on an implementation of that interface

hiredman18:01:05

for example I tend to have a *.protocols namespace where I put all my defprotocols,

joshkh18:01:46

hmm okay, i'll internalise that and see where i end up. thanks for your input, @hiredman

bfabry18:01:00

java is fine with circular dependencies (fine as in "compiles" not as in "good idea") so it makes sense for maven to be as well I suppose. I'm glad it produces a warning

hiredman18:01:35

clojure has a circular artifact dependency now, spec.alpha depends on clojure and clojure depends on spec.alpha

spacepluk19:01:51

hi clojurians, can you recomend a library/framework with low overhead to implement a very simple REST api? I've been looking into luminus, pedestal and immutant but I would love to hear about your experiences.

spacepluk19:01:16

@h.elmougy looks good! have you used it?

h.elmougy19:01:40

yes I have API in production using duct

h.elmougy19:01:05

it is great specially as it allows separating data from the logic

spacepluk19:01:25

I'm reading the docs, I'm liking what I see hehe

h.elmougy19:01:58

it is very simple and powerful. your gonna like it. enjoy 🙂

h.elmougy19:01:56

you can also configure async routes without adding external libraries

h.elmougy19:01:30

just configure the adapter to async

spacepluk19:01:11

great! thanks 🙂

bpiel20:01:10

I've used liberator many times over the years with success. I'd be interested in knowing whether Liberator and Duct could play nicely.

bostonaholic20:01:03

it’s very full-featured, and helps you stick to proper http/rest according to the spec

spacepluk20:01:49

awesome guys, with that I have to read for a while 🙂

jgh20:01:19

I like compojure+immutant

mchampine21:01:42

Just used https://github.com/metosin/compojure-api which is very simple, easy to use and gets you up and running instantly with swaggerized APIs. Just “lein new compojure-api myapi”

ghadi21:01:37

Anyone accessed the OS X Keychain to grab usernames/passwords from Clojure or the JVM? There is a old java repo out there that wraps the Keychain

phronmophobic22:01:14

@ghadi, I have, but I use jna since I’m already using that for other things

qqq22:01:27

@mikera: does core.matrix support strides for gemm ? I have an input matrix that 'self overlaps'

zalky22:01:03

Hi all: curious if anyone has ever tried to use strings with clojure's adhoc hierarchies. As far as I know, you can only use types that implement clojure.lang.Namedwith clojure's adhoc hierarchies, and I don't think you can extend java.lang.String since clojure.lang.Named is not a protocol, but an interface. So, is there no way to use strings with Clojure adhoc hierarchies? Something like (derive h "child" "parent")?

alexmiller22:01:49

No, most people use keywords for hierarchies

misha22:01:03

00 ;;=> 0
01 ;; => 1
07 ;; => 7
08 ;; clojure.lang.LispReader$ReaderException: java.lang.NumberFormatException: Invalid number: 08
¯\_(ツ)_/¯

eriktjacobsen22:01:49

numbers starting with 0 are octals, of which only 0-7 are valid values

misha22:01:05

(type 07) ;;=> java.lang.Long

petterik23:01:41

user=> 011
9

vemv23:01:21

@misha a Long expressed in octal notation 🙂 no issue here

eriktjacobsen23:01:52

numbers exist irrespective of what base they are displayed in

xiongtx23:01:27

@alexmiller Is there a way to add e.g. Artifactory as a repo for deps.edn? It’s not clear which repositories what :mvn/version accesses. I assume at least Maven and Clojars

alexmiller23:01:14

Yeah, you can add repos in a :mvn/repos key

alexmiller23:01:38

See ~/.clojure/deps.edn

alexmiller23:01:48

There’s an example there