Fork me on GitHub
#babashka
<
2020-11-20
>
helios13:11:30

what's the best way of generating a hmac-sha1 using bb?

borkdude13:11:12

@helios This question came up recently. The crypto class for that is currently missing, but the person solved this by shelling out to openssl

Travis Jefferson13:11:20

Hey that was me!

helios13:11:28

👋 do you have an example?

Travis Jefferson13:11:35

(defn sign-with-string [key msg]
  (-> (sh "openssl" "dgst" "-sha256" "-mac" "hmac" "-macopt" (str "key:" key) :in msg)
      :out
      str/trim))

(defn sign-with-hex [key msg]
    (-> (sh "openssl" "dgst" "-sha256" "-mac" "hmac" "-macopt" (str "hexkey:" key) :in msg)
        :out
        str/trim))

Travis Jefferson13:11:03

I needed sha256 but I believe swapping in -sha1 is as easy as it sounds

borkdude13:11:04

We should add docs for this

Travis Jefferson13:11:57

annoying bit for me was knowing which bits support binary data and which are limited to string

Travis Jefferson13:11:54

you can pass bytes to sh with :in, which is cool, but I couldn’t figure out a way to pass bytes in for a command option/flag (I had binary data that I wanted to use for a key)

Travis Jefferson13:11:15

so I ended up hex-encoding my binary keys and passing as a string with sign-with-hex

Travis Jefferson13:11:09

there are probably ways to work around, perhaps by spinning up a file to hold the binary key, or maybe some fancy shell tricks to pipe in binary data

Travis Jefferson13:11:19

but this was good enough for me

borkdude13:11:38

Since there are now two people needing this, https://github.com/borkdude/babashka/issues/656 Please post which classes / Clojure examples should be needed for this

borkdude13:11:35

We could also make a small babashka.crypto namespace (or some other name) with util functions

Travis Jefferson13:11:59

posted an example! it’s java code 🙈

Travis Jefferson13:11:23

making classes available and (optionally) providing a small utility ns both sound solid to me

Travis Jefferson13:11:33

at a glance, buddy-core’s APIs look nice; adding that might be a good longer-term goal if it’s reasonably compatible I suspect @borkdude would have the best intuition for what’s a good fit or not 😁

helios13:11:14

i also posted some more clojure code (that uses java)

helios13:11:23

a little offtopic, and what about base64 encoding? 😄

borkdude14:11:46

@helios

(deftest Base64-test
  (is (= "babashka"
         (bb nil "(String. (.decode (java.util.Base64/getDecoder) (.encode (java.util.Base64/getEncoder) (.getBytes \"babashka\"))))"))))

helios14:11:59

thanks 🙂

borkdude14:11:16

I think I prefer the babashka.crypto namespace. There we can collect functions that are most frequently used for scripting. Instead of including buddy which can grow babashka bigger than necessary. Also offering babashka.crypto will help swapping out different implementations in the future.

mkvlr15:11:28

can also shell out to shasum if it’s just about hashing…

shasum -a 256 tests.edn                                                                                                                                              
c0050533cb04ed9b5b3fc851b062830bac1c0c8f5569a3e4cec8bca9bb6dd54f  tests.edn

helios08:11:52

@U5H74UNSF it's about getting the hmac-sha1 of a string given a specific key for API signature 😢

helios08:11:58

but i got my answer 😄

skuro13:11:52

so I have this problem at hand: I've a 1.3G json file with stuff I need to feed an ElasticSearch instance with. The json itself contains a huuuuge JSON array. I'd like bb to lazily parse such array, chunk the contained objects into smaller batches then feed each batch to ES for ingestion

skuro13:11:05

seems to me that this is not possible with bb tho, correct me if I'm wrong

skuro13:11:51

as in, when I try to reach into the internals of cheshire which I need to lazily parse the array, it fails to resolve names / classes (even the basic jackson classes)

borkdude14:11:52

@skuro afaik cheshire has an option to parse an array lazily, but I'm not sure if this works for an array that is embedded into something else

borkdude14:11:26

for this amount of data it's probably better to just use Clojure on the JVM

borkdude14:11:02

If the top-level object is an array, it will be parsed lazily (use `parse-strict' if strict parsing is required for top-level arrays.

skuro15:11:39

oh, I see... I did json/parsed-seq but json/parse seems be be the only one doing lazy stuff

borkdude15:11:48

parsed-seq parses multiple top-level values

borkdude17:11:33

Is anyone using the smile functions from cheshire? I think I will remove them for the next release to free up 200kb from the binary, since I haven't seen anyone use them and I also don't use them myself. If there are any objections, I'll revert.

nate17:11:45

no objections from me, I haven't used the smile functions ever, even in jvm clojure

Dig21:11:01

same here never needed them

nate17:11:41

I'm curious, has anyone tried running buddy in babashka yet? I have need for doing some crypto stuff with it, and it would be convenient to be able to use bb

nate17:11:56

specifically, buddy.core.mac buddy.core.codecs and buddy.core.nonce, to create an HMAC signature to pass to a curl call

borkdude17:11:29

it's funny that you mention this! hmac has been coming up two times earlier this week. it's in the air :) https://github.com/borkdude/babashka/issues/656

borkdude17:11:54

could you post in that issue what specifics you would need?

nate17:11:44

cool, I'll try and make a small example

borkdude17:11:07

note that currently you could shell out to openssl and shasum.

nate18:11:47

commented with my JVM code, will comment again with my openssl-calling code, if I can get it to work :crossed_fingers:

nate19:11:27

Haha. Haven't had a spare moment yet. First chance is tomorrow evening.

borkdude19:11:44

No worries! :)

yubrshen22:11:03

I'd like to learn to use babashka to make http request. If you have some sample code or reference, would you please share. Thanks!

yubrshen23:11:14

@borkdude Thanks! I'll try them.