Fork me on GitHub
#clojure
<
2019-12-20
>
borkdude10:12:06

Can someone explain me how and if this should work in Clojure?

(let [{a :a
       {b :b} (name a)}
      {:a :a
       "a" {:b :b}}]
  (println "value of b" b))

borkdude10:12:20

(it was reported by a user of clj-kondo)

borkdude10:12:02

It seems the bound value of a is used in the "second" binding, but since it's a map destructuring, the bindings aren't ordered maybe, so I'm puzzled if this would work at all

yuhan10:12:04

whoa, that definitely smells like undefined behaviour, I had no idea that was valid syntax

yuhan10:12:40

Here's a more minimal example:

(let [{a :a, b a} {:a :x, :x :y}]
  b)
;; => :y

yuhan10:12:26

But reordering the keys in the destructuring form breaks it:

(let [{b a, a :a} {:a :x, :x :y}]
  b)
;; Syntax error compiling at (src/repro.clj:4:1).
;; Unable to resolve symbol: a in this context

p-himik10:12:27

(let [{_x  :a
       b1 :a
       c :a
       d :a
       e :a, f :a, g :a, h :a, i :a
       {b :b} (name _x)}
      {:a :a
       "a" {:b :b}}]
  (println "value of b" b))

Syntax error compiling at (form-init191682565753033842.clj:6:15).
Unable to resolve symbol: _x in this context

p-himik10:12:25

When you create a map using {}, it creates an array map if the number of items is <= 8. Otherwise, it creates a hash map. With array map, the order and hence the behavior is well defined. With a hash map - no so much.

borkdude10:12:04

This also doesn't seem to work:

(let [{a :a, b a, c b} {:a :x, :x :y}] c)
nil

p-himik10:12:49

I'm confused:

(let [{a :a, b a, c b, d c, e d} {:a :x, :x :y}]
  [a b c d e])
=> [:x :y nil nil nil]

p-himik10:12:54

How did it get to :y?

p-himik10:12:21

Oh wait, nevermind.

p-himik10:12:38

(this goes to demonstrate that such chaining can be confusing)

borkdude10:12:02

and undefined probably

p-himik10:12:43

Yeah, given your example, I retract my previous words about an array map resulting in a well defined behavior.

yuhan10:12:52

it makes more sense looking at the macroexpanded form of let , which is just a "dumb" syntactical transform that doesn't know about symbol resolution

yuhan10:12:01

I think clj-kondo is right to flag such things with at least a warning

borkdude10:12:22

yeah, might be nice

yuhan10:12:03

(as in, the current behavior already seems right)

p-himik10:12:07

But the (let [{a :a, b a, c b} {:a :x, :x :y}] c) is actually correct to return nil. Because there's no :y key in the map.

borkdude10:12:24

(let [{a :a, b a, c b} {:a :x}] c)

p-himik10:12:52

Same, absolutely correct. There's no :x key in the map.

p-himik10:12:25

Issues with such destructuring only arise when there are more than 8 items. So OK, I retract my retraction. :D

borkdude10:12:26

(let [{a :a, b a, c b} {:a :x}] a)
:x 

p-himik10:12:36

Yep, all correct.

borkdude10:12:14

Relying on the orderedness of maps seems bad to me

p-himik10:12:17

That's true. Maybe it's fine in some cases, but it seems unnecessary for the simple case of destructuring. It can be written in a not much less concise form without relying on anything. And it will also be more readable.

Linus Ericsson12:12:51

If I understand the code correctly, the destructuring goes from "right to left" in the given code, but that, I think, should be regarded as an implementation detail. One place where that potentially matters in when you destroy two things to the same variable. That is problematic but at least not more surprising than anything else. One way to give a actionable insight from clj-kondo could be to highlight if variables are destructed to twice, that seems like it should be treated like a mistake to me. Also if you rely "destruction targets" in the destruction expression, that's probably a mistake as well.

p-himik10:12:40

Wow, these examples are fun. My brain keeps turning back to them thinking that I still didn't get something.

borkdude10:12:37

Maybe a good reason to avoid it altogether

๐Ÿ‘ 4
bronsa10:12:47

undefined behavior

bronsa10:12:00

it's not supposed to work

borkdude10:12:17

Thanks for confirming

Alex Miller (Clojure team)12:12:49

agreed, you should not rely on this working

Alex Miller (Clojure team)12:12:24

there is actually a ticket about this I think

Alex Miller (Clojure team)12:12:14

I did not find a ticket specifically about this (not involving :or)

vemv16:12:29

WDYT of the following pattern for authoring macros that work in clj + cljs + self-hosted cljs?

vemv16:12:35

(defmacro foo [compilation-target]
  (case compilation-target
    :clj  :clj
    :cljs :cljs))

(foo #?(:clj  :clj
        :cljs :cljs))

4
vemv16:12:39

The targeted problem being: determining within the macro what the compilation target is - clj or cljs? I normally use (-> &env :ns nil?) but that's not an official API

p-himik16:12:06

Why not use the #?(...) within the macro itself?

p-himik16:12:44

I've never tried it - won't it work in a cljc file?

vemv16:12:10

Won't work as intended: for vanilla cljs, macros run in the JVM and reader conditionals will resolve to their :clj branch

p-himik16:12:28

Right. thheller in his article about macros uses exactly what you use normally - (if (:ns &env) %cljs-branch% %clj-branch%): https://code.thheller.com/blog/shadow-cljs/2019/10/12/clojurescript-macros.html#gotcha-4-clj-macros

vemv16:12:43

Yeah been using it for a while but I hate it :) I feel like it'll break at some point

p-himik16:12:38

Dunno, seems like everybody uses it. And Clojure[Script] is not really infamous for any backward incompatible changes.

p-himik16:12:35

But I get what you're saying - I cannot find any official documentation on it. Everyone just says "it works, that's how it is".

vemv16:12:46

I do perceive in Clojure core a commitment to not break APIs, and no particular commitment to not break non-APIs (which is quite reasonable, and sets a precedent to not rely on random hacks)

vemv16:12:15

For completeness - there's this answer reflecting it's an open problem https://stackoverflow.com/a/42209528/569050 Its suggested solution isn't that reliable - a JVM repl may have required that cljs.analyzer ns for "reasons". Has happened to me

vemv16:12:51

It's a bit noisy for consumers, but clean for the macro itself

vemv16:12:44

(haven't tried it yet in self-hosted cljs btw - elsewhere it appears to work correctly)

vemv17:12:19

tried with Lumo! works as expected.

samoleary17:12:52

hello, has anyone used GitHub Actions to run tests on a clojure project with leiningen, where a couple of the project dependencies are in a private S3 bucket? locally i've been using s3-wagon-private and aws access/secret keys in a ~/.aws/credentials file. i was hoping something like this would do the trick:

name: Clojure CI
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v1

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: eu-west-1

      - name: Install dependencies
        run: lein deps

      - name: Run tests
        run: lein test
but lein deps returns a Could not transfer artifact ... Access Denied (Service: Amazon S3; Status Code: 403; i've added the aws access/secret keys to the github repo, Settings -> Secrets, but i feels like i'm missing a step somewhere

ghadi17:12:09

have you tested the exact same policy locally?

ghadi17:12:50

if you can't use the same credentials, you can assume a role with the same policy and give it a shot locally

ghadi17:12:32

or add a call to aws sts get-caller-identity after you configure credentials to make sure everything is ๐Ÿ‘Œ:skin-tone-4:

ghadi17:12:32

(you may have to delete a private jar from ~/.m2/repository to make sure that lein re-pulls it down locally)

samoleary17:12:59

good call yeah, i've wiped the local repo in ~/.m2/repository, used the same access/secret key pair locally and have gotten a successful download

ghadi17:12:57

compare aws sts get-caller-identity locally and in Github to eliminate doubt

๐Ÿ‘ 4
samoleary17:12:41

github has masked the Account number, ***, but the UserId and role part of the Arn match up

ghadi17:12:48

weird -- which wagon plugin specifically are you using?

samoleary17:12:49

[s3-wagon-private "1.3.1"] i could bump that to the latest, 1.3.3 and see what happens

ghadi17:12:27

do you have :no-auth true set in the :repositories section?

samoleary17:12:28

:repositories configured with :no-auth true as well in project.clj

samoleary17:12:34

๐Ÿ˜‚ :thumbsup:

ghadi17:12:04

I'm fresh out of ideas, and we've debugged scientifically

ghadi17:12:11

I think bumping to 1.3.3 is in order

ghadi17:12:31

(I use the same exact setup but on 1.3.1 ๐Ÿ™‚ )

Alex Miller (Clojure team)17:12:59

the s3 region is of critical importance when accessing s3 buckets - you had "eu-west-1" above - is that where the bucket is?

ghadi17:12:57

aws s3api get-bucket-location  --bucket YOURBUCKET

samoleary17:12:37

{
    "LocationConstraint": "eu-west-1"
}

samoleary17:12:19

haven't bumped the version yet, i'll run that

ghadi17:12:24

not gonna work

ghadi17:12:31

I'm putting money on it

samoleary17:12:45

nah, it didn't ๐Ÿ˜‚ i was clutching at straws

ghadi17:12:04

can you paste the config and redact the bucket?

:plugins           [[s3-wagon-private "1.3.1"]]
  :repositories [["releases" {:url           ""
                              :no-auth       true
                              :sign-releases false}]])

ghadi17:12:08

mine looks like that ^

samoleary17:12:01

yep, mine's the mirror image

ghadi17:12:30

s3p proto?

ghadi17:12:45

another thing to debug is assume the same exact role then:

aws s3 cp --recursive  /tmp

๐Ÿ‘ 4
samoleary17:12:33

there's something else at play somewhere... fatal error: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

ghadi17:12:08

you try that locally or remotely?

ghadi17:12:17

shouldn't matter, just curious

samoleary17:12:21

that was within github actions

ghadi17:12:55

does the policy look just like that ? ^

samoleary18:12:54

ah, there's no policy on the S3 bucket

samoleary18:12:13

access to the bucket isn't being managed how i thought it was

samoleary18:12:05

thanks for your help @U050ECB92, i've no fear that the github action itself and the project.clj should work at this stage. i'll have to do my homework on how we're managing access to that private bucket, but for now it's beer o'clock on a friday evening. thanks again ๐Ÿป

ghadi18:12:29

โ˜ฎ๏ธ ๐Ÿป

ghadi17:12:55

^ that shouldn't fail inside github actions

snoe17:12:01

Hi, I'm working on adding support for schema.core/defn to clojure-lsp. An example call is

(s/defn stamped-names :- StampedNames
  [names :- [s/Str]])
Can anyone think of other libraries' macros with a similar approach to annotating parameters? Basically looking for other examples where a "parameter" vector would need to ignore some inner elements.

dpsutton17:12:57

I was thinking of ghostwheel. Not sure how widely used it is though

wilkerlucio20:12:26

ghostwheel is kind abandoned, but there its successor guardrails

flyboarder20:12:48

Reading the clojure.walk api docs, and I came across this:

Note: "walk" supports all Clojure data structures EXCEPT maps
created with sorted-map-by.  There is no (obvious) way to retrieve
the sorting function.
My initial thought is that the sorting function could be stored as meta-data on the map and used later? Just a thought.

๐Ÿ‘ 4
andy.fingerhut20:12:10

The Java implementation has a field storing the sorting function, so via appropriate kind of field access it is straightforward. It isn't published as a Clojure-level access function that I know of.

andy.fingerhut20:12:22

Do you have a link to that text you found?

flyboarder20:12:53

Overview section at the top

didibus22:12:04

@flyboarder Not sure what that means, but in a quick REPL test, it seems that walk supports sorted-map-by

didibus22:12:31

user=> (clojure.walk/postwalk-demo (sorted-map-by > 1 10 2 20))
Walked: 2
Walked: 20
Walked: [2 20]
Walked: 1
Walked: 10
Walked: [1 10]
Walked: {2 20, 1 10}
{2 20, 1 10}
user=> (clojure.walk/postwalk-demo (sorted-map-by < 1 10 2 20))
Walked: 1
Walked: 10
Walked: [1 10]
Walked: 2
Walked: 20
Walked: [2 20]
Walked: {1 10, 2 20}
{1 10, 2 20}

lilactown22:12:07

it might be that it doesnโ€™t preserve the order when you transform nodes?

noisesmith22:12:22

at least nominally postwalk-demo is transforming nodes, it just happens to return the same node it received

lilactown22:12:52

ostensibly you could change the value of something in place and it would not re-sort it?

noisesmith22:12:44

when I check the type returned by the postwalk, it's a clojure.lang.PeristentTreeMap

noisesmith22:12:53

I'll try a version that inverts sign of the keys

noisesmith22:12:17

(ins)user=> (clojure.walk/postwalk (fn [x] (if (number? x) (- x) x)) (sorted-map-by > 1 10 2 20))
{-1 -10, -2 -20}
(ins)user=> (type *1)
clojure.lang.PersistentTreeMap
that looks right to me

noisesmith22:12:57

it makes sense, if this works, then postwalk should work

)user=> (let [m (sorted-map-by >)] (into m {1 :a 2 :b 3 :c}))
{3 :c, 2 :b, 1 :a}

noisesmith22:12:43

relevant clause of clojure.walk/walk cond form:

(coll? form) (outer (into (empty form) (map inner form)))

lilactown22:12:43

then I have no idea what the docstring means

noisesmith23:12:24

it might be the doc is stale on that point?

noisesmith23:12:39

it calls into, into preserves your sort function