Fork me on GitHub
#clojure
<
2019-08-30
>
eccentric J01:08:59

If you are querying elements for each item in list, but you want the first element that matches a predicate. What would you use? I could do this with map, filter, and first but that means every element would be processed.

seancorfield01:08:19

@jayzawrotny If you really want the first matching element and want no others inspected, I'd probably reduce and stop with reduced.

seancorfield01:08:50

map/`filter` won't process "every" element but would likely process up to 32 more than you needed (chunked sequences).

eccentric J01:08:23

Right in my case list is < 32 but the DOM processing may be expensive.

seancorfield01:08:54

(reduce (fn [_ x] (when (pred x) (reduced x))) nil elements)

😮 8
Lone Ranger19:08:58

I didn't realized you could break out of a reduce early with reduced... that's kind of a game-changer

seancorfield20:08:08

Since Clojure 1.5 (March 2013).

eccentric J01:08:06

I didn't know about reduced, that's awesome!

eccentric J02:08:53

Realized I was over-engineering it and could use a take-while instead. But seems to be working!

seancorfield02:08:52

Often simple is best. Until you know you have a performance issue. Then optimize 🙂 @jayzawrotny

eccentric J02:08:43

True, I realized the heavy lifting can be done in the predicate which should make it decently performant. Technically it will be doing more work as you scroll down but it should be fine for a first draft prototype with < 10 items. If it does slow down, then perhaps my approach here itself is wrong.

dominicm08:08:38

@jayzawrotny there's a function in medley for this that is well optimized

vlaaad08:08:20

isn't it what some is for?

vlaaad08:08:54

(some #(when (pred %) %) coll)

vlaaad08:08:43

and if elements in your coll can be falsey, you can write it as (first (some #(when (pred %) [%]) coll))

👍 4
eccentric J01:08:04

Interesting. I did not realize some would return anything other than true or false. Now I understand why it doesn’t end with a ? Thanks!

Leon09:08:39

Is (eval (apply list [+ 1 1])) supposed to work? Storing the Form in a vec instead of a list makes it so that the + function isn't just '+, but actually the function Object. My eval works like this.... Until i include spec instrumentation anywhere. Then it Breaks, saying something like "no ctor found for spec..." is this a Bug in spec or am i using unstable behaviour of eval?

vlaaad09:08:59

@lkowarschick don't know about spec, but I think it should be (eval (apply list ['+ 1 1])) (notice quote)

vlaaad09:08:52

eval is supposed to work on forms that have symbols instead of functions in them.

Leon09:08:49

yea quoting the function makes sense i guess ^^ but it should somehow be possible to use it without, seeing that i.e. reagent uses [function args] in its hiccup-style html syntax,...

vlaaad09:08:37

reagent does not eval functions, it applies them

vlaaad09:08:26

(let [f+args [+ 1 2 3]]
  (apply (first f+args) (next f+args))) => 6

Leon09:08:49

ahhh maybe thats an option for me too, thanks, ill try it ;D

noisesmith12:08:58

@lkowarschick as an aside to all the above, I'd never use (apply list [x y z]) instead of (list x y z) - the result will always be the same

noisesmith12:08:42

and with eval, as with macros, ` is likely more useful

noisesmith12:08:29

(ins)user=> (let [x 1] (eval '(+ x x)))
Syntax error compiling at (REPL:1:19).
Unable to resolve symbol: x in this context
(cmd)user=> (let [x 1] (eval `(+ ~x ~x)))
2

noisesmith12:08:57

alternatively

user=> (let [x 1] ((eval '(fn [y] (+ y y))) x))
2

naveen13:08:37

Can someone help me to use generator for multi-spec .. I have looked all over .. but couldn't find anything so far

naveen13:08:26

@noisesmith following those examples generates spec ony with the tag key

noisesmith14:08:29

when I run the last gen/sample repeatedly, I see both variants, :example-key and :different-key

noisesmith14:08:44

which means it's generating both methods

noisesmith14:08:06

it provides the :tag because that's part of the spec

noisesmith14:08:20

perhaps I am misunderstanding

naveen14:08:45

@noisesmith thanks ... I had an issue with spec itself .. I used ::req-un and after changing it to :req-un (with one less colon) it works

escherize15:08:52

I'm curious... why do some libraries use the com.orgname.somethingelse naming scheme, and some just use projectname? Is one preferable over the other for some reason?

noisesmith15:08:30

one convention is more correct according to the host ecosystem, the other is preferred by people who think the host ecosystem are overengineering things

escherize15:08:37

Why is this overengineering? just wondering

noisesmith15:08:25

in most languages you can call your project "foo" and publish it with that name, it's more of a java thing to expect everyone to namespace projects based on a unique URI they own

noisesmith15:08:22

sonatype coordinates are a different issue - I thought you were talking about package names

noisesmith15:08:35

(the underlying concerns are of course similar)

noisesmith15:08:17

leiningen encourages the looser convention, even allowing you to call a project "foo", and automatcally publishing the artifact as "foo/foo" and generating a "foo.core" ns

noisesmith15:08:02

where if you use lein new org.noisesmith.foo it would do the correct thing and publish as "org.noisesmith/foo" and create the "org.noisesmith.foo" ns

noisesmith15:08:11

(but few people use that style)

escherize15:08:44

https://clojars.org/search?q=group-id%3Acom.* seems like many folks use this scheme. Is this mostly a convention inherited from java?

noisesmith15:08:11

that's what I mean by "the host ecosystem" - we use jvm deps, and publish jars

noisesmith15:08:26

yes, it's the standard java convention

noisesmith15:08:16

I think you'll find that most clojure libs don't use that convention though (they just don't all get packed into one directory like com / org / net do)

escherize15:08:44

Alright, I've been using the looser convention, and will continue to do so. (e.g. https://github.com/escherize/tracks )

escherize15:08:51

thanks for the chat @noisesmith

dominicm15:08:08

You'll be at more risk of conflicts by doing so. Fyi.

dominicm15:08:39

I've seen conflicts happen in clojure, nobody ends up happy.

noisesmith15:08:55

right, there's no way to use two namespaces called tracks.core, one wins, the other is unavailable

noisesmith15:08:59

and if you leave your project.clj as is, the dep manager will ensure only one tracks/tracks is pulled in (a single segment project name is repeated twice by lein to generate a group/artifact pair)

dominicm16:08:10

So there's two conflict spaces here, and they're separate because reasons.

Alex Miller (Clojure team)16:08:40

my perspective is that the prefix of your group-id should be something you "control", either by dns, trademark, or otherwise

👍 4
zane16:08:23

I've seen people recommend that you use a domain you control, or your GitHub username. My GitHub username is just zane but using that as the first segment of my group-id feels a bit presumptuous.

zane16:08:06

Thanks for the external validation. 🙂 I have a library I've been holding off on publishing because I've been anxious about the finality of decisions around things like naming and versioning.

Alex Miller (Clojure team)16:08:24

I was just agreeing that deciding on naming and versioning sucks :)

Alex Miller (Clojure team)16:08:51

and is often a hard gate in the way of actually getting things done :)

zane16:08:11

Ah, got it. 🙂

dominicm16:08:45

The other is getting the name you want on maven central and others and ensuring its unique. The dns system is a good way to go about this.

dominicm16:08:19

Things get a bit more ambiguous for applications, I think you still want a good bit of prefix though.

Alex Miller (Clojure team)16:08:42

if you're not publishing something to a public repo, do whatever you want

Alex Miller (Clojure team)16:08:49

the common practice on clojars of using same group-id and project-id is not a good practice per standard Maven practices (they won't let you do that in Maven central, although they've grandfathered in older things)

Alex Miller (Clojure team)16:08:09

I wish we had a more established convention of "borrowing" identity from 3rd parties (in addition to dns/trademark stuff)

4
Alex Miller (Clojure team)16:08:22

so for example, github identity could confer sufficient uniqueness

Alex Miller (Clojure team)16:08:54

so like a project with coordinate github.puredanger/foo

zane16:08:36

Yeah, I've seen people just use their GitHub handles as-is.

Alex Miller (Clojure team)16:08:47

(note that this then conflicts with github itself publishing its own things to maven central)

Alex Miller (Clojure team)16:08:08

which is why a better convention would be helpful

4
Alex Miller (Clojure team)16:08:37

I'm not going to solve that problem today :)

dominicm16:08:01

Only if they name it after you!

dominicm16:08:31

I do receive reluctance to use the top level domain on things, but less so trademarks. Company names seem to be considered unique enough, and not so tedious to type that it's okay.

kenny16:08:10

How would I decode a b64-encoded binary input stream to a plaintext string? I tried data.codec but I can't seem to get it to work. Here's what I'm trying to do:

(let [r (aws-api/invoke c {:op      :GenerateDataKey
                           :request {:KeyId   cmk-arn
                                     :KeySpec "AES_256"}})
      plaintext (:Plaintext r)]
  (prn plaintext)
  (with-open [p plaintext
              out (io/output-stream "test.txt")]
    (clojure.data.codec.base64/decoding-transfer p out)))
#object[java.io.BufferedInputStream 0x477586db "[email protected]"]
java.lang.ArrayIndexOutOfBoundsException: Index -92 out of bounds for length 123

hiredman16:08:13

oh, I see you already tried data.codec, I would look at test.txt and see if actually is base64 encoded

hiredman16:08:31

that error suggests it isn't, or is some other variant of base64

kenny16:08:30

test.txt is empty. Hard to tell because the base64 is binary.

hiredman16:08:50

sorry, not paying close enough attention, check plaintext

kenny16:08:51

I mean, not in a string form I can see haha

kenny16:08:22

slurping plaintext is a binary string

hiredman16:08:33

base64 encoding, basically the reason it was created, was to encode arbitrary binary as printable ascii characters

hiredman16:08:01

so if you are getting arbitrary binary data and not printable ascii it is not base64 encoded

kenny16:08:10

Ah. It's "�$���r*�!��8yS�x;�\f�@\r�[email protected]�"

kenny16:08:01

I think something else is going on because I doubt the AWS docs would be totally wrong. It says plaintext is > Base64-encoded binary data object

manutter5116:08:29

Maybe you have some middleware that’s automatically converting it from B64 to binary before you get it?

kenny16:08:03

Hmm, perhaps the cognitect aws-api is doing something

kenny16:08:36

Typically it doesn't really do much of anything though

kenny16:08:34

Asked in #aws since it may be an aws-api lib issue. It would surprise me though. I'm guessing that weird string needs to be converted from whatever that is to regular b64.

manutter5116:08:19

Just scanning through the github repo I can’t work out exactly how that’s used, but it’s there.

noisesmith16:08:25

for a moment I wondered if base62 existed :D

kenny16:08:39

Hahaha same. Had to check the source

andy.fingerhut18:08:01

We should MAKE one!!

noisesmith16:08:08

reminds me of a very silly thing I made - base-;lkjfdsa https://gist.github.com/noisesmith/561c215ffa39f71645b5

noisesmith16:08:46

user> (base-_SEMI_lkjfdsa-encode "hello")
";ddks;jjfddas;;;"
user> (base-_SEMI_lkjfdsa-decode ";ddks;jjfddas;;;")
"hello"

manutter5116:08:49

That looks like fun

noisesmith16:08:58

it was a cute exercise to figure out

manutter5116:08:36

“NSA AGENT: The suspect seems to be sending coded messages. OTHER NSA AGENT: Nah, that’s probably just line noise.”

dpsutton16:08:56

does that compile arbitrary binary into valid vim commands?

manutter5117:08:07

Doesn’t everything?

💯 4
noisesmith17:08:07

in certain subcultures "alksdf;laksdjf;lkajsdfl;jasdl;" is equivalent to "LOL" combined with "I can't even"

noisesmith17:08:13

so it hides in plain sight

noisesmith17:08:32

txt-speak keyboard mashing steganography

kenny17:08:46

Maybe that string is actually what I'm supposed to get returned. I was kinda expecting a regular rsa key or something of the likes.

noisesmith17:08:38

@kenny it could be the raw version of the hash, where you can construct the normal printed representation via %02x format

noisesmith17:08:27

eg. this one liner to get the sha-256 string

user=> (let [digest (java.security.MessageDigest/getInstance "sha-256") input (.getBytes "hello" "UTF-8") result (.digest digest input)] (->> result (map #(format "%02x" %)) (apply str)))
"2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

noisesmith17:08:05

you'd just need the last part (->> result (map ...) (apply str)) to get the "hex" output from the bytes of the string

noisesmith17:08:19

oh, you'd want .getBytes before that

noisesmith17:08:45

user=> (->> (.getBytes "hello" "UTF-8") (map #(format "%02x" %)) (apply str))
"68656c6c6f"

kenny17:08:50

@noisesmith I did this:

(let [digest (java.security.MessageDigest/getInstance "sha-256")
      r (aws-api/invoke c {:op      :GenerateDataKey
                           :request {:KeyId   key-arn
                                     :KeySpec "AES_256"}})
      plaintext (:Plaintext r)]
  (->> (slurp plaintext) (map #(format "%02x" %)) (apply str)))
java.util.IllegalFormatConversionException: x != java.lang.Character
I'm now fairly certain that weird string is the expected return value facepalm I followed this guide (https://dev.to/matchilling/pragmatically-storing-security-sensitive-data-using-aws-kms-5e5b) to get the decoded, plaintext key and it looks very similar to what I received.

noisesmith17:08:07

sorry my example was lazy

noisesmith17:08:39

you don't want to calculate the digest, you just want the last part of turning a byte-array into hex string - see my more recent example

noisesmith17:08:34

or were you coincidentally already using sha-256...

kenny17:08:51

Oh that did something interesting haha. I got "efbfbd2cefbfbd18efbfbd5f1b5956d1926b084e5befbfbd38efbfbdefbfbdefbfbdefbfbdefbfbdefbfbd30efbfbd40efbfbd0e77efbfbd13efbfbd"

noisesmith17:08:21

I'm not sure if that's the form you need, but at least it is a format that round trips, as opposed to the weird string you had before

4
noisesmith17:08:11

it could be you want the base64, so you could re-encode it to that format

noisesmith17:08:23

it really depends on what you are trying to do with this payload I guess

borkdude20:08:25

can I use *in* in combination with tools.reader?

(with-in-str "(+ 1 2 3)"
    (edn/read *in*))
;;=> Pushback buffer overflow
If not, how can I get from *in* to something that tools.reader can work with?

borkdude20:08:06

the use case for me is that clojure.server rebinds *in* to a reader that reads from a socket and I want to do my own thing with this. clojure.server doesn't expose the connection directly

hiredman20:08:39

you should checkout the tools.reader readme maybe

borkdude20:08:20

@hiredman are you suggesting the answer is indeed in the README? because I had skimmed it before asking the question here 🙂

seancorfield20:08:33

(require '[clojure.tools.reader.reader-types :as t])
;=> nil
(def reader (t/string-push-back-reader "1"))

seancorfield20:08:49

(from the readme)

borkdude20:08:56

that's not an anwer to my question though?

seancorfield20:08:17

Are you trying to read a string with pushback?

borkdude20:08:34

no, I'm trying to read from what clojure.server gives me as *in*

seancorfield20:08:53

(t/push-back-reader *in*) maybe?

borkdude20:08:19

hmm, that did work

seancorfield20:08:54

There's a whole bunch of push back readers in that ns.

borkdude20:08:22

I didn't yet think of that since LineNumberingPushbackReader is already a ... PushbackReader 🙂

borkdude20:08:37

but probably not one that tools.reader can work with