Fork me on GitHub
#clojure
<
2017-02-08
>
qqq00:02:35

Is there any notion of 'caching' a list of spec predicates that succeeds on a given object? Suppose I hve a spec that is recursive, I don't want it to manually chec for each subtree whether a certain property holds every time -- I want it to chec once, and then somehow tags it

bfabry00:02:48

@qqq these may be better in #clojure-spec, anyway, if I wanted to optimise a particular predicate for repeat calls with the same input then I would memoize it

johnnyillinois00:02:04

Does anyone have any code to proxy all method invocations of a java object with some middleware? Something like:

(defn log [m & args} (println "I am logging") (.invoke m args))

(proxy-all jdbc-connection log)

johnnyillinois00:02:22

Continued Example ...

(def p (proxy-all jdbc-connection log))

(.commit p)
"I am logging"
nil

joshjones01:02:10

@johnnyillinois you should check out the robert hooke library https://github.com/technomancy/robert-hooke .. does pretty much exactly what you want

hiredman01:02:52

hooke works with clojure functions, but java objects are a different kettle of fish

hiredman01:02:27

you can often use reflection (java.lang.reflect.*) and a macro or two to do that sort of thing

johnnyillinois02:02:44

Yeah I was hoping to not roll my own but hey open source library train here I come

eslachance04:02:19

hmm. what do I do if I have code that starts up events and a websocket, and it... well when I try to run lein uberjar it actually runs my program and won't stop 😄

seancorfield05:02:04

You have things defined with def instead of defn -- so it executes when namespaces are loaded.

seancorfield05:02:26

(and namespaces are loaded when you run lein uberjar because of AOT)

eslachance05:02:10

I'm guessing what I need to do is require some sort of argument when running. or just accept that I have to execute a specific function when running it?

seancorfield05:02:23

Avoid def 🙂

eslachance05:02:16

right. is there any way to tell clojure to start something when I lein run it (or run the jar) but to not run it when I create the jar? Would that be the main function I delete by reflex because Im a noob 😛

seancorfield05:02:03

Sure, only call functions from inside -main. Don't use def.

eslachance05:02:16

Lesson learned. As usual, thanks Sean 😄

seancorfield05:02:51

You can have a namespace with a -main for lein run and have a different namespace with a -main for the JAR created by lein uberjar.

seancorfield05:02:44

What I tend to do is have the -main for the uberjar and then have a (comment ...) with various expressions that I can evaluate to define my application component, start it, stop it).

seancorfield05:02:26

for example...

eslachance05:02:45

Hmm. ok. Now, when I try to run lein uberjar it creates it but tells me

Warning: The Main-Class specified does not exist within the jar. It may not be executable as expected. A gen-class directive may be missing in the namespace which contains the main method. 

eslachance05:02:59

oh nm it's right it doesn't have gen-class

seancorfield05:02:16

Per @hiredman you really don't need a gen-class.

seancorfield05:02:19

We don't AOT anything. We build an uberjar. We run the jar and tell it what the namespace is containing the -main function.

eslachance05:02:59

alright, well it works with lein run so I just gotta try it with the jar now ^_^

seancorfield05:02:01

java -jar the-jar-file.jar -m our.main.namespace

eslachance05:02:54

hmm.

D:\home\devel\dtbot\target\uberjar>java -jar dtbot-0.1.0-SNAPSHOT-standalone.jar
Error: Could not find or load main class dtbot.core

seancorfield05:02:17

Note: my example specified -m to run the main namespace

eslachance05:02:29

Tried that, and it was the exact same error however

seancorfield05:02:32

Same as lein run (if you don't specify it in project.clj)

eslachance05:02:42

lein run works actualy

seancorfield05:02:41

The default -- if you don't specify :main in project.clj is to run clojure.main which is the default entry point, which accepts the -m argument to specify the namespace that contains the actual -main to run.

eslachance05:02:00

right but the main namespace is (ns dtbot.core

seancorfield05:02:15

Right, remove :main from project.clj

seancorfield05:02:29

Then you can specify it at runtime.

eslachance05:02:33

So remove :main ^:skip-aot dtbot.core then

seancorfield05:02:13

@hiredman has a PR in to the Clojure site repo to update the docs to show this way of doing things because it totally avoids all the AOT stuff

eslachance05:02:45

oh there we go 😄

eslachance05:02:42

Oh boy. now here's a doozy. Tried to connect my lib (which uses websockets) to a.. let's say "bigger" account. io.netty.handler.codec.CorruptedFrameException: Max frame length of 65536 has been exceeded. is this something I can... uhm... "fix" by allowing bigger packets? I'm using aleph/manifold

eslachance05:02:08

That would probably be due to receiving packets that contain... oh, about 6000 keys that are users, and a few other hundred various bits and pieces like channels and permissions and roles and emojis and all that jazz. Think, oh, 1631655 characters is a "small" packet.

eslachance05:02:17

(it comes in compressed with zlib though)

eslachance06:02:47

it's a stupid question but what does 65536 represent? bytes?

sonelliot06:02:41

that's the capacity of a 2-byte value I believe

bfabry06:02:07

maximum unsigned integer representable in 2 bytes

sonelliot06:02:26

you probably need to do some kind of packet fragmentation to work-around the limit

sonelliot06:02:31

I have no experience with either aleph or netty though sorry 😐

ilevd08:02:39

Does anyone know how to insert multiple rows via YESQL? Is it possible?

pesterhazy11:02:47

(printf "%s" (pr-str #"..."))

pesterhazy11:02:23

printf or format don't know about clojure's print mechanism (it's just java.lang.String/format under the hood)

ska11:02:38

Ah, printf goes directly to format which probably calls .toString while print (in the end) dispatches via print-method which has an impl for java.util.regex.Pattern (https://github.com/clojure/clojure/blob/master/src/clj/clojure/core_print.clj#L393). Get it.

roelof11:02:00

some experts who can help me in the beginners channel with how to mock up a service in spec

tjtolton15:02:41

okay, so im trying to create a function called upmerge to help me with configuration management. the idea is that, given a nested map, i want to push values up from the root into the leaves, with the caveat that anything specified closer to the leaves takes precedence. example input output:

tjtolton15:02:47

I cant figure out quite how to make this work or even describe it.

tjtolton15:02:36

rather, I can think of how to make it work as a recursive function, but it seems like it would be better written using a simple function passed to "merge with"

schmee15:02:24

tjtolton this sounds like a job for Specter

tjtolton15:02:07

I've heard that repo's name get thrown around

schmee15:02:38

it is awesome

schmee15:02:07

if you ask your question in the #specter channel there’s a good chance that the man himself will answer your question 🙂

triss16:02:53

hey all. would somebody be able to help me translate a curl command line to something that wil work with clj-http?

triss16:02:03

Command is

curl --basic -u "myFirstApp":"dsLE57s1wXRodpJqTYpUkUfY2S0iEv4i" --header "x-temboo-domain: /triss/master" --header "Content-Type: application/json" --header "Accept: application/json" -X POST --data '{
"inputs": [	{"name":"Query", "value":"Clojure"}]
}' ""

triss16:02:33

I can see how to encode the headers. It’s the -u and —data I’m unsure how to encode

pesterhazy16:02:03

check the basic-auth and body parameters

triss16:02:37

I see how to use the basic-auth option

triss16:02:17

I’m still scratching my head re: encoding the —data in to the :body paramater though

geoffs17:02:39

what do you mean encoding the data @triss? Your curl command is saying that the content type is JSON, and the body is JSON. If you have a JSON string, just put that in the :body parameter. If you have a clojure data structure, you need to turn it into a JSON string, probably with clojure.data.json or cheshire or something

geoffs17:02:55

you also want to make sure to still set the Content-Type 🙂

triss17:02:27

ag thanks @geoffs was just looking at data.json

triss17:02:36

and can see how to produce the query

geoffs17:02:43

:thumbsup:

josh_tackett17:02:14

Anybody do slack webhook integrations? Code (http/post "https://hooks.slack.com/services/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" {:debug true :form-params {:payload (json/encode {"text:" “TEST”})}}) Error status: 500 body: missing_text_or_fallback_or_attachments

aamosnyc17:02:32

@josh_tackett I think if you send json you want to set the content-type header to application/json

aamosnyc17:02:08

@josh_tackett also I think you want your payload in the post body, not in the form params

josh_tackett17:02:52

it was the : after text in string

aamosnyc17:02:59

ah yeah, just saw that

josh_tackett17:02:01

other than that the request was correct

josh_tackett17:02:10

form-params is correct

aamosnyc17:02:41

right, form-params generates a body, my mistake!

jdkealy18:02:57

is it possible to give an alias to keys in destructured args? e..g (defn do-something [{:keys [key1 :as key2]}])

rafaelzlisboa18:02:35

@jdkealy you can do so if you use the other syntax for destructuring (the one that doesn’t use :keys)

rafaelzlisboa18:02:16

(defn do-something [{key1-alias :key1 key2-alias :key2}])

jdkealy18:02:14

awesome thanks!

devn18:02:48

clojure.lang.PersistentQueue, discuss.

aengelberg18:02:20

devn: I am persistently queue-rious.

danielstockton19:02:55

I wonder if anyone is good with java streams, channels, bytes and byte-arrays. I'm having trouble with https://gist.github.com/danielstockton/a4ab9204a8d8806a95bd3c491ed932ca Trying to write some data out with fressian gzipped and then read it back. Currently, the fress/read line gives a EOFException. I would like to use the basic java API rather than a library.

geoffs19:02:04

I would invert the way you're trying to do that. Don't mess with a channel so you can write a bytebuffer to it.

geoffs19:02:27

it looks like data.fressian has an API for passing an OutputStream I would think you can just pass the GZipOutputStream to that, and then get transparent compression and fressian writing

geoffs19:02:05

you would need to use data.fressian/create-writer and data.fressian/write-object instead of the write convenience method

danielstockton20:02:08

Thanks, I'll look into those methods 👍

danielstockton20:02:21

(def write-handlers
  (-> fress/clojure-write-handlers
      fress/associative-lookup
      fress/inheritance-lookup))

(def read-handlers
  (-> fress/clojure-read-handlers
      fress/associative-lookup))

(def out
  (let [baos   (java.io.ByteArrayOutputStream.)
        baos   (GZIPOutputStream. baos)
        writer (fress/create-writer baos :handlers write-handlers)]
    (fress/write-object writer {:bob :jim})
    (.toByteArray baos)))

(let [bais   (java.io.ByteArrayInputStream. out)
          bais   (GZIPInputStream. bais)
      reader (fress/create-reader bais :handlers read-handlers)
      out    (fress/read-object reader)]
  out)
Something like this. I'm getting an exception because of .toByteArray not found on class GZIPOutputStream.

danielstockton20:02:38

Ha, funnily the read part works on some data I had already defined in my repl session.

geoffs20:02:44

it's because you're shadowing the name. use a new name for the GZIPOutputStream You actually don't even need to keep a reference to the GZIPOutputStream, you could pass it in directly to create-writer

danielstockton20:02:19

You're right, just got it working.

(def write-handlers
  (-> fress/clojure-write-handlers
      fress/associative-lookup
      fress/inheritance-lookup))

(def read-handlers
  (-> fress/clojure-read-handlers
      fress/associative-lookup))

(defn compress [data]
  (let [baos   (java.io.ByteArrayOutputStream.)
        gzip   (GZIPOutputStream. baos)
        writer (fress/create-writer gzip :handlers write-handlers)]
    (fress/write-object writer data)
    (.close gzip)
    (.close baos)
    (.toByteArray baos)))

(defn decompress [data]
  (let [bais   (java.io.ByteArrayInputStream. data)
        bais   (GZIPInputStream. bais)
        reader (fress/create-reader bais :handlers read-handlers)
        out    (fress/read-object reader)]
    out))

(decompress (compress {:bob :jim}))

danielstockton20:02:42

I need a reference to close it, I think.

danielstockton20:02:15

Thanks a lot for the help.

geoffs20:02:38

usually wrappable Input or Output streams can just be closed by the top level wrapper. Also, since this bottoms out in a ByteArray, it's not super critical to close it (no associated file handle to be leaked)

geoffs20:02:00

and the FressianWriter closes whatever real writer you pass to it when you close it https://github.com/Datomic/fressian/blob/master/src/org/fressian/FressianWriter.java#L479

danielstockton20:02:13

Hmm, if I omit the .close calls I get 'unexpected end of ZLIB input stream'

danielstockton20:02:56

It looks like I only need to close the GZIP stream, not the baos

geoffs20:02:40

that makes sense. ZLIB probably has a footer-like structure and calling close causes that to get written :thumbsup:

whilo20:02:58

i am having a problem with protocols and defrecord that i don't understand

whilo20:02:04

tests pass

whilo20:02:22

i get "No implementation of method: :-serialize of protocol: #'konserve.protocols/PStoreSerializer found for class: konserve.serializers.FressianSerializer"

whilo20:02:24

i have tried to import the record, but only evaling the form (in the REPL) of (defrecord FressianSerializer ...) helps

whilo20:02:15

the code then runs

whilo20:02:00

my problem is that i don't understand how the protocol implementations are exactly loaded and why they are missing in twitter-collector

jr20:02:04

try adding [konserve.serializers] to your ns require

whilo20:02:50

i have done so, even then (:impls PStoreSerializer) returns nil

whilo20:02:40

even after evaling the form in fact

whilo20:02:47

which i don't understand

hiredman20:02:17

how are you loading the code?

hiredman21:02:01

most likely you are (re)loading the protocol definitions after loading the record definitions, effectively defining a new protocol with the same name

hiredman21:02:39

almost certainly that is the case

hiredman21:02:15

oh, actually, the other related possibility is you are importing the record jvm types without requiring the namespace that defines the records first

chriscraws21:02:43

Hey clojurians, do you guys have advice for how to navigate clojure.core from the ide? I'm using CIDER, but my best bet still seems to be googling for examples and reading docs that group functions by what abstractions they deal with - i.e. seq, coll, etc

whilo21:02:25

@hiredman the code is linked above, but i just required the protocol parts where i needed them. does this recreate protocol definitions? i am unclear about how to understand what is loaded when and why. is there a simple way to find out?

hiredman21:02:50

calling require will load it if it has not been loaded, calling require with :reload or :reload-all cause things to be reloaded, if you are working with an editor with a send to repl feature or eval-buffer or whatever, and you run on that on the protocol code, that will reload it

hiredman21:02:10

if you reload the protocol definitions, and reload the record definitions, if you still have instances of the previous record definitions in memory somewhere, they will still satisfy the old protocol, not the one you just redefined

whilo21:02:27

i understand that. the konserve namespaces work even in a project where i use stuart sierras component with tools.namespace reload

whilo21:02:41

it seems to be something specific about this project and its loading order

whilo21:02:56

but in general i am curious how to debug these things (from the REPL preferably)

hiredman21:02:41

I start with a fresh repl, that doesn't have you code loaded in to it

hiredman21:02:04

require the namespace that defines the records, construct a record, call one of your protocol functions on it

hiredman21:02:57

that will work fine, but it is good to verify

hiredman21:02:55

then kill that repl, put a (println "FOO") at the top of the file where the protocols are defined, then do whatever you did where you saw the error before, you will see FOO printed out twice

hiredman21:02:26

then look at what happened between each time FOO was printed

hiredman21:02:21

somewhere in the middle is something that reloaded the protocol definitions, but not the record definitions, and either they both need to be reloaded or neither

whilo22:02:06

FOO turns up once in the repl server buffer

whilo22:02:30

i will instrument the defrecord as well

hiredman22:02:10

are you aot compiling this project?

whilo22:02:57

well, there is a gen-class in the main core.clj, but i load it in the REPL

hiredman22:02:16

when was the last time you blew away target/ ?

joshjones22:02:37

@chriscraws I use intelliJ, and press F12. Mouse -> and <- map to navigating through the code. I would think emacs has a way to jump to a c.c function definition though, if that's what you're using.

whilo22:02:10

@hiredman the instrumented version seems to have updated the classpath

whilo22:02:19

suddenly it works

whilo22:02:29

the error went away

hiredman22:02:35

blow away target/, and stop aot compiling

hiredman22:02:08

likely when you changed the file that gave it a newer timestamp than the class file in target/

hiredman22:02:22

which indicates you are aot compiling, and you have stale classes in there

whilo22:02:26

yes, but these bugs still haunt me

whilo22:02:11

i am doing clojure 5 years now, but this one cost me another day

whilo22:02:25

and now it was just an invalid cache, seemingly

hiredman22:02:34

it is aot compilation

hiredman22:02:40

people need to stop doing it

jr22:02:45

aot = bad

hiredman22:02:49

I have to tell people every day

whilo22:02:57

i do not like to do it. i just needed a callable main

whilo22:02:06

so i used :gen-class

whilo22:02:25

@hiredman what about creating a jar file?

hiredman22:02:28

that is the most common reason people give for aot compiling, and it is not required

whilo22:02:30

that is why it is there

hiredman22:02:47

aot compilation has nothing to do with jar files

hiredman22:02:10

you should read the above linked thing

chriscraws22:02:29

@joshjones: I can jump through c.c ok, what i'm more looking for is a solid way to understand the relationships between the functions defined there, ideally without having to google for the information. for example, in the online clojure docs each has a 'related pages' section and tells me which functions perform what kind of tasks - is there any such organization implicitly in c.c?

whilo22:02:53

thanks @hiredman ! 🙂 i didn't know about the -m switch. i hope the discouragement of gen-class gets more popular

whilo22:02:31

i will promote your document

joshjones22:02:42

@chriscraws well, you'll find namespace-related functions mostly grouped together, transient-related functions together, etc., but i don't think what you're looking for is going to be done in such a way in any code base, because the relationships between functions is not 1:1 and so well defined. You'll see logical groupings, but not a hyperlinked type of system