Fork me on GitHub
#clojure
<
2018-02-28
>
benzap02:02:38

@caleb.macdonaldblack this doesn't really answer your question, but you should check out https://github.com/nathanmarz/specter

benzap02:02:18

It was built specifically to solve querying and transforming of deeply nested data structures

lwhorton03:02:26

i bumped into a hiccup with file i/o and laziness and was hoping someone could point me in the right direction: I’m using a reader to (.readLines reader) into a lazy-seq, something like

(let [rdr (io/reader (io/resource path))]
    (letfn [(read-next []
              (lazy-seq
                (if-let [line (.readLine rdr)]
                  (cons line (read-next))
                  (do (.close rdr) nil))))]
      (read-next))))
And from these lines I’m attempting to lazily process the strings
(-> line
      (clojure.string/trim)
      ;; clean up leading/trailing space around newlines
      (clojure.string/replace #"\s*\n+\s*" " ")
      ;; use consistent \space char
      (clojure.string/replace #"\s" " ")
      ;; split into codepoints (chars as strings)
      (clojure.string/split #"")))
and eventually reduce them into a different shape and spit them out into another file. My intent was to make this whole process lazy so as to avoid pulling everything into memory. What I’m hoping for is essentially a lazy sequence of cleaned, trimmed, and char-split strings like ["h" "e" "l" "l" "o" ...] that I can run through a reducer. What i’m getting instead (and I cannot seem to figure out why) is entire lines at a time like ["h e l l o ..."]. Do i have a foundational misunderstanding of io as it relates to laziness somewhere? It feels like somehow I need to “merge” the individually consumed lines so they’re read as a single string instead of a seq of strings, but I’m not sure how do do that without realizing the entire lazy seq?

hagmonk03:02:39

@caleb.macdonaldblack AFAIK you still can't use clojure.walk/postwalk because of the bug @seancorfield filed: https://dev.clojure.org/jira/browse/CLJ-2031

seancorfield03:02:12

@lwhorton Your code seems to work fine for me...

lwhorton03:02:51

Hmm. So you are getting a proper ["h" "e" "l" "l" "o"] and not a ["h e l l o"]?

lwhorton03:02:32

It might very well be that my repl and forced eval from printing is tricking me into thinking something is off, but I don’t think so. The final step in my process is to zip and reduce two seqs:

(reduce
    (fn [prev [i [a b]]]
       ...
      )
    []
    (map-indexed vector (map list seq-a seq-b))))
But for some reason the zipping doesn’t work as expected because the “lines” of the io seem to be stuck together, and string splitting doesn’t break them apart into a continuous seq of chars, but rather a continuous seq of long strings broken into chars

seancorfield03:02:04

I did

(for [line ( ... (read-next) )]
  (-> line ... your pipeline ...))
and got a sequence of sequences. Each sub-sequence has single-character strings as desired.

lwhorton04:02:29

hmm, thanks for confirming that i’m going crazy, ill keep debugging

blackawa05:02:40

Does anyone know a function xxx which does this?

user=> (xxx [a '(1 2 3) b '(2 3 4)] {:a a :b b})
;; '({:a 1 :b 2}
;;   {:a 2 :b 3}
;;   {:a 3 :b 4})

gonewest81805:02:08

it’s a list comprehension. for

blackawa05:02:34

Thank you @gonewest818!

user=> (for [a [1 2 3] b [2 3 4]] {:a a :b b})
;; ({:a 1, :b 2} {:a 1, :b 3} {:a 1, :b 4} {:a 2, :b 2} {:a 2, :b 3} {:a 2, :b 4} {:a 3, :b 2} {:a 3, :b 3} {:a 3, :b 4})
However, this return value is not what I want… Do I use for wrong way…?

curlyfry06:02:36

@blackawa If you supply multiple collections to map it does what you want:

(map (fn [a b] {:a a :b b}) [1 2 3] [2 3 4])
=> ({:a 1, :b 2} {:a 2, :b 3} {:a 3, :b 4})

gonewest81815:02:36

Ah, good catch. I thought this was one of those koans, where the form other than the xxx was a fixed problem definition. Focused on something that took the vector of bindings, not the output.

blackawa07:02:00

@curlyfry Thanks! This is what I want!

mbjarland09:02:02

with the risk of starting a bar fight, would anybody care to enumerate some of the upsides/tradeoffs when choosing between component, mount, and integrant (or others)?

gnejs10:02:14

from my very subjective view I think Integrant feels more “functional” and “simple”.

sundarj11:02:39

@mbjarland guess i should have linked to the relevant part 😁

mbjarland11:02:34

@U61HA86AG wow...still reading through the passionate debate in that thread : ) what a fun read...and also what a testament to the clojure community that we can have a philosophical disagreement about things we are passionate about without devolving into cave trolls

curlyfry11:02:19

@mbjarland My poorly researched and also very subjective view is that: - Component is nicer than Mount in that it is more easily composable and testable + doesn't overload the namespace construct. - Integrant has similar upsides to Component, but is more data-oriented, and imo simpler + easier to understand.

benzap11:02:16

I originally started using Component, but switched to mount mainly since component requires more tie-in and mount feels less like a framework and more like a tool for managing state

benzap11:02:36

I'd say either one is fine, I guess I just prefer the simplicity of mount over component.

mschaef11:02:10

Any thoughts on how to identify the the point where it starts to make sense to consider either?

benzap11:02:55

Hard to say... I found that mount had a much leaner barrier of entry, and it made more sense to me. It leverages it's dependency management through clojure namespaces, so it feels closer to home

benzap11:02:16

Component on the other hand wrote it's own dependency management scheme, and uses a complex map to represent it's state

benzap11:02:26

I'm not familiar with integrant

benzap11:02:34

Component on the other hand represents it's lifecycle in a much cleaner way, the main differences are that Component leverages protocols to represent state. If you typically use Records and Protocols to represent your data, you'd love Component

benzap11:02:08

however, if you are more interested in a data-oriented approach with the base datatypes, go with mount

benzap11:02:45

again, I don't know anything about integrant, I believe it came later, so maybe it has some advantages over the other two

mbjarland11:02:53

/ listening in - this is exactly the kind of discussion I was hoping for. Very useful /

benzap11:02:49

Glad I could help 🙂

mschaef13:02:41

Thanks… I appreciate the advice… I’ll play around a bit with a few and see where I wind up.

xtreak2911:02:24

Is there a standard way to say whether the java version is 9 or 8 or 7. Closest I can think of is (Integer/parseInt (first (clojure.string/split (System/getProperty "java.version") #"\."))) . I want to run some tests only on JDK 8 and others on JDK 9 . In python's pytest we can mark the tests to be run only for certain versions of python. Do I need to roll this out on my own ? I am sure there will be some way since this is a common problem.

Jakub Holý (HolyJak)12:02:49

@alexmiller not sure where/how to report this but search at http://clojure.org seams broken - https://clojure.org/search?q=destructuring returns no results

polymeris14:02:12

Hi. I am working on a clojure library that's now compatible with the most popular (python) implementation of a standard. However I would also like to add the option to enable compatibility with the RFC. The implementation difference is trivial, however I am not sure how to expose the option to switch between both. Would you: * use a dynamic var that can be set to :python-compatible or :rfc-compatible? * pass the option explictely to the library's functions? * duplicate the functions (there are only 2), creating python- and rfc-compatible versions? maybe in separate namespaces?

polymeris14:02:47

Also should I default to one of the two, or throw if none is selected?

mschaef14:02:29

IMHO, option 3. Consumers can then pick the namespace they need for the behavior they want.

mschaef14:02:40

(Or bring in both if they need both for some reason.)

mschaef14:02:28

But if you already have something like keywords args or an option map, maybe option 2 (b/c in that case, it’s just another key/value pair)

polymeris14:02:06

Thanks, I was also leaning towards 3. The only problem I see is that any dependant libraries would also have to create 2 namespaces if they want to keep the flexibility. And, no, I don't have an options map yet, I just pass data to the functions.

mschaef14:02:35

Some of it will come down to a judgement about how important you think the ability to change this option is. The namespace approach implies a bit of a willingness to commit more strongly to one or the other of the two option values at a given call site…

mschaef14:02:34

(Stronger still, you could do something like two modules, and clients would pick the dependency they prefer.)

mschaef14:02:21

The only other thing I’d suggest is that you do what you can to make information on this setting obvious… (docs, good error messages if you can, etc.)

polymeris14:02:29

Makes sense. And I agree about the documentation. Probably more important than the decision itself. Thanks!

owen19:02:12

does anyone use http-kit? if so have you ever had an issue with {:as :stream} seemingly leaving file handles open in /proc/?

tanzoniteblack19:02:57

we recently migrated away from http-kit as an HTTP client in favor of clj-http. We found that clj-http had much better throughput for high amounts of parallelism, where as http-kit will start throwing connection timeouts if you make too many simultaneous requests. And with either of these, you have to make sure to close the stream you get back in the body

owen19:02:45

@U236LQYB0 I prefer clj-http as well

owen19:02:50

and it was what we were using before

owen19:02:09

I couldn't overcome a TLS issue that was resolved immediately by using http-kit so I made the jump

owen19:02:12

I am closing the stream

owen19:02:49

but isn't this now how streams are supposed to work?

owen19:02:01

(.close stream)
(.read stream)
=> 126

owen19:02:06

shouldn't it not be readable after closign?

owen19:02:10

I thought that threw an exception

tanzoniteblack19:02:49

I got nothing. If you have a minimum example, file an issue in GitHub?

owen20:02:26

😞 ok thanks!

hiredman19:02:23

are you closing the streams?

owen19:02:18

althought my java is slightly rusty

owen19:02:23

shouldn't this throw an exception?

owen19:02:32

(.close stream)
(.read stream)

owen19:02:36

I am able to do that with the http-kit stream and read from it after calling close

hiredman19:02:10

sounds like a bug?

owen20:02:39

thats what I was afraid of 😞 thanks!

schmee20:02:16

@bhauman sorry for calling you out of lurk, but I saw that you were active in another room so might as well give it a shot 🙂 I’m really eager to try rebel-readline but I can’t figure out how to get it into and make it replace the default REPL in an existing Leiningen project. any help to get started or pointers in the right direction would be much appreciated 🙏

bhauman20:02:25

@schmee Add it to your lein dependencies, lein trampoline run -m rebel-readline.main

bhauman20:02:45

I'm going to add that info to the readme shortly

leira20:02:41

question about destructuring, it seems they are not lazy.

leira20:02:50

but this version didn't work, it is not lazy~

leira20:02:08

seems like the problem is in [p & lst], seems & is not lazy

bhauman20:02:19

it is lazy

hiredman20:02:24

your types don't match

hiredman20:02:20

every time you recursively call sieve you are adding another layer of seq nesting to lst

hiredman20:02:35

(for sieve-d)

hiredman20:02:11

my guess is you meant [[p & lst]] there

pesterhazy20:02:22

destructuring simply desugars to first/next:

(macroexpand-1 '(let [[a & b] [1 2 3]]))
(let* [vec__156 [1 2 3] seq__157 (clojure.core/seq vec__156) first__158 (clojure.core/first seq__157) seq__157 (clojure.core/next seq__157) a first__158 b seq__157])

schmee20:02:30

@bhauman well, that was easy, thanks! 😄 is there a way to make vi mode the default? pressing ctrl+x,ctrl+v works but as soon as I press enter it reverts back to the regular bindings

pesterhazy20:02:34

so it cannot possibly make a difference

leira20:02:59

yes!! rookie mistake~

leira20:02:01

thanks guys~!!

hiredman20:02:23

I would recommend taking a closer look at the error messages you are getting

bhauman20:02:40

@schmee in ~/.clojure/rebel_readline.edn {:key-map :viins}

leira20:02:06

I will next time~ though what I observed was, it started to print an infinite list...

bhauman20:02:07

probably going to be the two most common questions

bhauman20:02:56

@schmee I edited the last one to remove the '.'

schmee20:02:57

haha yeah, answering these two questions early in the readme will probably save you many, many hours 😅

Alex Miller (Clojure team)20:02:17

can you set print-length by default?

bhauman20:02:29

@alexmiller not in the config

bhauman20:02:40

but with clojure.main -i opts

Alex Miller (Clojure team)20:02:48

I mean can you just default it to something

bhauman20:02:10

you mean as the library author can I supply a default?

Alex Miller (Clojure team)20:02:11

I guess that’s really not in the “read” area

Alex Miller (Clojure team)20:02:32

sorry, I’m probably not enough in the loop to be saying sensible things :) so I will stop

bhauman20:02:49

in fact the default main is just there for trying it out

schmee21:02:08

@bhauman well, now I’ve been using it for almost a full minute, and all I can say is ohmygodthisthingisamazing

schmee21:02:32

this is going to make Clojure coding so much more enjoyable for me! 😄

bhauman21:02:22

@alexmiller if you ever have any questions please let me know, but the rebel-readline.main can only really be launched from clojure.main and, it should have all the default root bindings etc.

bhauman21:02:04

also the rebel-readline.main right now is more of an example of what can be created with rebel-readline

bhauman21:02:23

I could make if a proxy for clojure.main, but at the moment it's not so simple to do that

Alex Miller (Clojure team)21:02:41

@bhauman just actually trying it for the first time and this is amazing stuff

bhauman21:02:08

well thanks 🙂

bhauman21:02:52

just trying to bring Clojure to the people

Alex Miller (Clojure team)21:02:05

I really would love to see if we could include this by default with clj

bhauman21:02:57

@alexmiller you could just add it add as dep and call a different main

schmee21:02:14

how do you intend to distribute documentation? in the readme? github wiki?

bhauman21:02:36

readme to start

schmee21:02:39

I want to start customizing the keybindings, and since I’ve already done some lein-related stuff as well it would be nice to have a place to write it down so other can benefit

Alex Miller (Clojure team)21:02:36

I think it would be ideal if we bundled and included it in the clojure tools uberjar, would need to think about the main question though. the clj script can do whatever.

bhauman21:02:04

@schmee its not a magical land of customize the bindings how you see fit... Jline has some hard dependencies on some of the keybindings behavior

bhauman21:02:34

for non core bindings though it should work great

bhauman21:02:56

@schmee for keybinding stuff lets' put it in the Wiki 🙂

schmee21:02:06

yeah, I’m thinking first and foremost of the inline source and doc commands

schmee21:02:25

my vim-attuned brain can’t handle all the ctrls 😂

Alex Miller (Clojure team)21:02:29

why would you need anything but emacs?

noisesmith21:02:33

oh yeah! that's a good idea

noisesmith21:02:51

@alexmiller my wrists beg to differ

Alex Miller (Clojure team)21:02:25

the trick is to bind all keys to control

pavani21:02:10

Is there a way to return an input stream from this function instead of writing it into a file? I cant separate the logic to write into a file in this function.

(defn encrypt-bytes
  [bytes dest ^PGPPublicKey pub-key]
  (let [enc   (doto (BcPGPDataEncryptorBuilder. PGPEncryptedData/AES_256)
                (.setWithIntegrityPacket true)
                (.setSecureRandom (SecureRandom.)))
        gen   (doto (PGPEncryptedDataGenerator. enc)
                (.addMethod (BcPublicKeyKeyEncryptionMethodGenerator. pub-key)))]
    (with-open [out (.open gen (io/output-stream dest) (long (alength bytes)))]
      (.write out bytes))))
https://github.com/thi-ng/crypto/blob/663a83148cc49cf5793eb7352dcd754b4dd174ab/src/thi/ng/crypto/core.clj#L152

pavani21:02:15

hey @U8XJ15DTK. Thank you for your response.I don't even want to write into any destination. Is there a way I can do that?

greglook21:02:32

which returns a byte array or an armored string

pavani21:02:32

The above library was my first choice. But the receiver was not able to decrypt my message. I used keyring/load-public-keyring to loan my pub key. Have you had a chance to use the clj-pgp library?

greglook21:02:23

Well I wrote it, so yes. 😛

greglook21:02:53

Make sure you’re using a subkey that is suitable for data encryption when you write the message.

greglook21:02:09

For example, just taking the first public key is unlikely to work.

pavani21:02:23

OMG 😞 I'm so sorry. I'm totally lost. I've been trying to get this done under so much pressure that I didn't notice. Thank you big time for responding to me. 😃 I have some questions regarding the library.

pavani21:02:52

I'll send you my code snippet in just a bit

greglook22:02:34

Sure, happy to help. 🙂

pavani22:02:56

(def keyring (keyring/load-public-keyring (io/file "something_public.asc")))
  (def pubkey (first keyring))
  (def data "hello world")
  (def message
    (pgp-msg/encrypt
      data pubkey
      :format :utf8
      :cipher :aes-256
      :compress :zip
      :armor false))
I ran the following and it said it cannot use DSA encryption. CompilerException java.lang.IllegalArgumentException: Can't use DSA for encryption.

greglook22:02:18

instead of (first keyring) try (->> keyring (keyring/list-public-keys) (filter (comp :encryption-key? pgp/key-info)) first)

greglook22:02:55

if you do (map pgp/key-info (keyring/list-public-keys keyring)) in the repl you should see what this is trying to do, and why taking the first pubkey won’t work

pavani22:02:46

Oh right I see it now. I was able to encrypt the file. Thanks a ton. 🙏 You were very very very helpful and I love the library! 🙂

hiredman21:02:47

I would return a byte array

bhauman21:02:20

@schmee @noisesmith if there are better vim bindings for the clojure-behavior let me know in an issue we can discuss

hiredman21:02:26

like a baos

bhauman21:02:53

@schmee also don't change the indent binding 🙂

schmee21:02:22

@bhauman if you add some lein-stuff to the readme, make sure you mention how to put it into your ~/.lein/profiles.clj so that you can have the awesomeness available at all times 👍

schmee21:02:15

@bhauman is it possible to change the key binding via the config file? I’ve been looking around in the source a bit but I can’t seem to find anything

bhauman21:02:43

yeah this isn't possible yet you would have to construct your own repl

bhauman21:02:09

its possible to create a line-reader and then change the bindings and then use that for your repl, but probably better to ask me to change the vim bindings

schmee21:02:12

roger, having more “vim-y” binding for the stuff in bind-clojure-widgets would be neat, I’ll think about it a bit and let you know if I stumble on something that feels natural

schmee21:02:53

on a different note, I just had a baller idea: can I make a nodisassemble widget thing and plug that in somehow?

bhauman21:02:33

@schmee I'd start by experimenting with the core api, and then work your way up from there 😉

schmee22:02:21

@bhauman haha, I just spent 30 minutes trying to lein install the repo locally, and I just realized that between I discovered the repo an hour ago and I cloned it the artifact name changed! 😂

schmee22:02:43

well, at least I figured it out 😄

bhauman22:02:36

sorry about that

schmee22:02:42

no worries, pre-release software is called pre-release for a reason 🙂

logistark22:02:46

Has anyone experience with firebase-admin sdk? My aleph service crash when added.

schmee22:02:06

the syntax highlighting just works and everything, holy crap

bhauman22:02:06

@schmee oh send a PR for that!

schmee22:02:33

comin’ right up 👍

bhauman22:02:48

that wasn't too hard was it?

bhauman22:02:05

but I'm surprised you did that so fast ...

schmee22:02:14

rebel-readline/project.clj                                |  5 +++--
 rebel-readline/src/rebel_readline/clojure/line_reader.clj | 24 +++++++++++++++++++++---
 rebel-readline/src/rebel_readline/core.clj                |  2 +-
 3 files changed, 25 insertions(+), 6 deletions(-)

greglook22:02:17

exciting work!

greglook22:02:44

Does this do syntax highlighting on the output? Haven’t played with it yet, but would love to write the whidbey-equivalent integration for puget

greglook22:02:28

Actually, maybe this is orthogonal, since it’s using the clojure core repl under the hood. :thinking_face:

bhauman22:02:15

yeah its really a library that you can create REPLs with

bhauman22:02:26

that has a decent repl example in it

bhauman22:02:16

that being said it does color output

schmee22:02:42

@bhauman I can only commend you for a very well-written and easy to understand library 👏

bhauman22:02:11

@schmee that diff isn't expanding for me

bhauman22:02:35

or is that just partial

bhauman22:02:58

I get it you were just giving me the stats