This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-02-28
Channels
- # beginners (226)
- # boot (18)
- # bristol-clojurians (4)
- # cider (1)
- # clara (77)
- # cljs-dev (79)
- # cljsjs (27)
- # clojure (178)
- # clojure-austin (9)
- # clojure-dev (30)
- # clojure-gamedev (11)
- # clojure-italy (5)
- # clojure-losangeles (3)
- # clojure-poland (1)
- # clojure-spec (42)
- # clojure-uk (34)
- # clojurescript (182)
- # core-async (5)
- # core-logic (2)
- # cursive (17)
- # datascript (12)
- # datomic (33)
- # emacs (8)
- # figwheel (1)
- # fulcro (25)
- # jobs (6)
- # jobs-discuss (27)
- # lein-figwheel (1)
- # lumo (18)
- # off-topic (17)
- # onyx (5)
- # pedestal (7)
- # re-frame (30)
- # reagent (52)
- # remote-jobs (1)
- # ring (2)
- # ring-swagger (1)
- # shadow-cljs (40)
- # spacemacs (5)
- # sql (31)
- # unrepl (12)
- # vim (25)
@caleb.macdonaldblack this doesn't really answer your question, but you should check out https://github.com/nathanmarz/specter
It was built specifically to solve querying and transforming of deeply nested data structures
i bumped into a hiccup with file i/o and laziness and was hoping someone could point me in the right direction:
I’m using a reader to (.readLines reader)
into a lazy-seq, something like
(let [rdr (io/reader (io/resource path))]
(letfn [(read-next []
(lazy-seq
(if-let [line (.readLine rdr)]
(cons line (read-next))
(do (.close rdr) nil))))]
(read-next))))
And from these lines I’m attempting to lazily process the strings
(-> line
(clojure.string/trim)
;; clean up leading/trailing space around newlines
(clojure.string/replace #"\s*\n+\s*" " ")
;; use consistent \space char
(clojure.string/replace #"\s" " ")
;; split into codepoints (chars as strings)
(clojure.string/split #"")))
and eventually reduce them into a different shape and spit them out into another file. My intent was to make this whole process lazy so as to avoid pulling everything into memory.
What I’m hoping for is essentially a lazy sequence of cleaned, trimmed, and char-split strings like ["h" "e" "l" "l" "o" ...]
that I can run through a reducer. What i’m getting instead (and I cannot seem to figure out why) is entire lines at a time like ["h e l l o ..."]
.
Do i have a foundational misunderstanding of io as it relates to laziness somewhere? It feels like somehow I need to “merge” the individually consumed lines so they’re read as a single string instead of a seq of strings, but I’m not sure how do do that without realizing the entire lazy seq?@caleb.macdonaldblack AFAIK you still can't use clojure.walk/postwalk
because of the bug @seancorfield filed: https://dev.clojure.org/jira/browse/CLJ-2031
@lwhorton Your code seems to work fine for me...
It might very well be that my repl and forced eval from printing is tricking me into thinking something is off, but I don’t think so. The final step in my process is to zip and reduce two seqs:
(reduce
(fn [prev [i [a b]]]
...
)
[]
(map-indexed vector (map list seq-a seq-b))))
But for some reason the zipping doesn’t work as expected because the “lines” of the io seem to be stuck together, and string splitting doesn’t break them apart into a continuous seq of chars, but rather a continuous seq of long strings broken into charsI did
(for [line ( ... (read-next) )]
(-> line ... your pipeline ...))
and got a sequence of sequences. Each sub-sequence has single-character strings as desired.Does anyone know a function xxx
which does this?
user=> (xxx [a '(1 2 3) b '(2 3 4)] {:a a :b b})
;; '({:a 1 :b 2}
;; {:a 2 :b 3}
;; {:a 3 :b 4})
it’s a list comprehension. for
Thank you @gonewest818!
user=> (for [a [1 2 3] b [2 3 4]] {:a a :b b})
;; ({:a 1, :b 2} {:a 1, :b 3} {:a 1, :b 4} {:a 2, :b 2} {:a 2, :b 3} {:a 2, :b 4} {:a 3, :b 2} {:a 3, :b 3} {:a 3, :b 4})
However, this return value is not what I want…
Do I use for
wrong way…?@blackawa If you supply multiple collections to map
it does what you want:
(map (fn [a b] {:a a :b b}) [1 2 3] [2 3 4])
=> ({:a 1, :b 2} {:a 2, :b 3} {:a 3, :b 4})
Ah, good catch. I thought this was one of those koans, where the form other than the xxx was a fixed problem definition. Focused on something that took the vector of bindings, not the output.
with the risk of starting a bar fight, would anybody care to enumerate some of the upsides/tradeoffs when choosing between component, mount, and integrant (or others)?
@mbjarland see https://clojurians-log.clojureverse.org/clojure/2017-07-21.html#inst-2017-07-21T22:32:57.901156Z
@mbjarland guess i should have linked to the relevant part 😁
@U61HA86AG wow...still reading through the passionate debate in that thread : ) what a fun read...and also what a testament to the clojure community that we can have a philosophical disagreement about things we are passionate about without devolving into cave trolls
these are worth reading too: https://gist.github.com/pandeiro/9a1c8fd431e1b4c78c99 https://www.reddit.com/r/Clojure/comments/3zppdn/the_beauty_of_clojure/
@mbjarland My poorly researched and also very subjective view is that: - Component is nicer than Mount in that it is more easily composable and testable + doesn't overload the namespace construct. - Integrant has similar upsides to Component, but is more data-oriented, and imo simpler + easier to understand.
I originally started using Component, but switched to mount mainly since component requires more tie-in and mount feels less like a framework and more like a tool for managing state
I'd say either one is fine, I guess I just prefer the simplicity of mount over component.
Any thoughts on how to identify the the point where it starts to make sense to consider either?
Hard to say... I found that mount had a much leaner barrier of entry, and it made more sense to me. It leverages it's dependency management through clojure namespaces, so it feels closer to home
Component on the other hand wrote it's own dependency management scheme, and uses a complex map to represent it's state
Component on the other hand represents it's lifecycle in a much cleaner way, the main differences are that Component leverages protocols to represent state. If you typically use Records and Protocols to represent your data, you'd love Component
however, if you are more interested in a data-oriented approach with the base datatypes, go with mount
again, I don't know anything about integrant, I believe it came later, so maybe it has some advantages over the other two
/ listening in - this is exactly the kind of discussion I was hoping for. Very useful /
Thanks… I appreciate the advice… I’ll play around a bit with a few and see where I wind up.
just watched the presentation on integrant (https://skillsmatter.com/skillscasts/9820-enter-integrant-a-micro-framework-for-data-driven-architecture-with-james-reeves). Have to say it looks pretty promising
Is there a standard way to say whether the java version is 9 or 8 or 7. Closest I can think of is
(Integer/parseInt (first (clojure.string/split (System/getProperty "java.version") #"\.")))
.
I want to run some tests only on JDK 8 and others on JDK 9 . In python's pytest we can mark the tests to be run only for certain versions of python. Do I need to roll this out on my own ? I am sure there will be some way since this is a common problem.
@alexmiller not sure where/how to report this but search at http://clojure.org seams broken - https://clojure.org/search?q=destructuring returns no results
I’m aware
Hi. I am working on a clojure library that's now compatible with the most popular (python) implementation of a standard. However I would also like to add the option to enable compatibility with the RFC. The implementation difference is trivial, however I am not sure how to expose the option to switch between both.
Would you:
* use a dynamic var that can be set to :python-compatible
or :rfc-compatible
?
* pass the option explictely to the library's functions?
* duplicate the functions (there are only 2), creating python- and rfc-compatible versions? maybe in separate namespaces?
IMHO, option 3. Consumers can then pick the namespace they need for the behavior they want.
But if you already have something like keywords args or an option map, maybe option 2 (b/c in that case, it’s just another key/value pair)
Thanks, I was also leaning towards 3. The only problem I see is that any dependant libraries would also have to create 2 namespaces if they want to keep the flexibility. And, no, I don't have an options map yet, I just pass data to the functions.
Some of it will come down to a judgement about how important you think the ability to change this option is. The namespace approach implies a bit of a willingness to commit more strongly to one or the other of the two option values at a given call site…
(Stronger still, you could do something like two modules, and clients would pick the dependency they prefer.)
The only other thing I’d suggest is that you do what you can to make information on this setting obvious… (docs, good error messages if you can, etc.)
Makes sense. And I agree about the documentation. Probably more important than the decision itself. Thanks!
does anyone use http-kit? if so have you ever had an issue with {:as :stream}
seemingly leaving file handles open in /proc/
?
we recently migrated away from http-kit
as an HTTP client in favor of clj-http
. We found that clj-http
had much better throughput for high amounts of parallelism, where as http-kit
will start throwing connection timeouts if you make too many simultaneous requests.
And with either of these, you have to make sure to close the stream you get back in the body
@U236LQYB0 I prefer clj-http as well
I couldn't overcome a TLS issue that was resolved immediately by using http-kit so I made the jump
I got nothing. If you have a minimum example, file an issue in GitHub?
@bhauman sorry for calling you out of lurk, but I saw that you were active in another room so might as well give it a shot 🙂 I’m really eager to try rebel-readline but I can’t figure out how to get it into and make it replace the default REPL in an existing Leiningen project. any help to get started or pointers in the right direction would be much appreciated 🙏
every time you recursively call sieve you are adding another layer of seq nesting to lst
destructuring simply desugars to first/next:
(macroexpand-1 '(let [[a & b] [1 2 3]]))
(let* [vec__156 [1 2 3] seq__157 (clojure.core/seq vec__156) first__158 (clojure.core/first seq__157) seq__157 (clojure.core/next seq__157) a first__158 b seq__157])
@bhauman well, that was easy, thanks! 😄 is there a way to make vi mode the default? pressing ctrl+x,ctrl+v works but as soon as I press enter it reverts back to the regular bindings
so it cannot possibly make a difference
haha yeah, answering these two questions early in the readme will probably save you many, many hours 😅
can you set print-length by default?
@alexmiller not in the config
I mean can you just default it to something
I guess that’s really not in the “read” area
sorry, I’m probably not enough in the loop to be saying sensible things :) so I will stop
@bhauman well, now I’ve been using it for almost a full minute, and all I can say is ohmygodthisthingisamazing
@alexmiller if you ever have any questions please let me know, but the rebel-readline.main can only really be launched from clojure.main and, it should have all the default root bindings etc.
also the rebel-readline.main
right now is more of an example of what can be created with rebel-readline
I could make if a proxy for clojure.main, but at the moment it's not so simple to do that
@bhauman just actually trying it for the first time and this is amazing stuff
I really would love to see if we could include this by default with clj
@alexmiller you could just add it add as dep and call a different main
I want to start customizing the keybindings, and since I’ve already done some lein-related stuff as well it would be nice to have a place to write it down so other can benefit
I think it would be ideal if we bundled and included it in the clojure tools uberjar, would need to think about the main question though. the clj script can do whatever.
@schmee its not a magical land of customize the bindings how you see fit... Jline has some hard dependencies on some of the keybindings behavior
why would you need anything but emacs?
oh yeah! that's a good idea
@alexmiller my wrists beg to differ
the trick is to bind all keys to control
it’s a fun game
Is there a way to return an input stream from this function instead of writing it into a file? I cant separate the logic to write into a file in this function.
(defn encrypt-bytes
[bytes dest ^PGPPublicKey pub-key]
(let [enc (doto (BcPGPDataEncryptorBuilder. PGPEncryptedData/AES_256)
(.setWithIntegrityPacket true)
(.setSecureRandom (SecureRandom.)))
gen (doto (PGPEncryptedDataGenerator. enc)
(.addMethod (BcPublicKeyKeyEncryptionMethodGenerator. pub-key)))]
(with-open [out (.open gen (io/output-stream dest) (long (alength bytes)))]
(.write out bytes))))
https://github.com/thi-ng/crypto/blob/663a83148cc49cf5793eb7352dcd754b4dd174ab/src/thi/ng/crypto/core.clj#L152@U7MMSSQKC maybe https://greglook.github.io/clj-pgp/api/clj-pgp.message.html#var-encrypted-data-stream does what you want?
hey @U8XJ15DTK. Thank you for your response.I don't even want to write into any destination. Is there a way I can do that?
actually you probably want package
: https://greglook.github.io/clj-pgp/api/clj-pgp.message.html#var-package or encrypt
See the README for an example: https://github.com/greglook/clj-pgp#message-handling
The above library was my first choice. But the receiver was not able to decrypt my message. I used keyring/load-public-keyring to loan my pub key. Have you had a chance to use the clj-pgp library?
Make sure you’re using a subkey that is suitable for data encryption when you write the message.
OMG 😞 I'm so sorry. I'm totally lost. I've been trying to get this done under so much pressure that I didn't notice. Thank you big time for responding to me. 😃 I have some questions regarding the library.
(def keyring (keyring/load-public-keyring (io/file "something_public.asc")))
(def pubkey (first keyring))
(def data "hello world")
(def message
(pgp-msg/encrypt
data pubkey
:format :utf8
:cipher :aes-256
:compress :zip
:armor false))
I ran the following and it said it cannot use DSA encryption. CompilerException java.lang.IllegalArgumentException: Can't use DSA for encryption.
instead of (first keyring)
try (->> keyring (keyring/list-public-keys) (filter (comp :encryption-key? pgp/key-info)) first)
if you do (map pgp/key-info (keyring/list-public-keys keyring))
in the repl you should see what this is trying to do, and why taking the first pubkey won’t work
Oh right I see it now. I was able to encrypt the file. Thanks a ton. 🙏 You were very very very helpful and I love the library! 🙂
@schmee @noisesmith if there are better vim bindings for the clojure-behavior let me know in an issue we can discuss
@bhauman if you add some lein-stuff to the readme, make sure you mention how to put it into your ~/.lein/profiles.clj
so that you can have the awesomeness available at all times 👍
@bhauman is it possible to change the key binding via the config file? I’ve been looking around in the source a bit but I can’t seem to find anything
its possible to create a line-reader and then change the bindings and then use that for your repl, but probably better to ask me to change the vim bindings
roger, having more “vim-y” binding for the stuff in bind-clojure-widgets
would be neat, I’ll think about it a bit and let you know if I stumble on something that feels natural
on a different note, I just had a baller idea: can I make a nodisassemble widget thing and plug that in somehow?
@schmee I'd start by experimenting with the core api, and then work your way up from there 😉
@bhauman haha, I just spent 30 minutes trying to lein install the repo locally, and I just realized that between I discovered the repo an hour ago and I cloned it the artifact name changed! 😂
rebel-readline/project.clj | 5 +++--
rebel-readline/src/rebel_readline/clojure/line_reader.clj | 24 +++++++++++++++++++++---
rebel-readline/src/rebel_readline/core.clj | 2 +-
3 files changed, 25 insertions(+), 6 deletions(-)
Does this do syntax highlighting on the output? Haven’t played with it yet, but would love to write the whidbey-equivalent integration for puget
Actually, maybe this is orthogonal, since it’s using the clojure core repl under the hood. :thinking_face: