Fork me on GitHub
#clojure
<
2020-03-26
>
pedrorgirardi05:03:17

Does anyone know why this doesn’t work on Spec 2 - or more likely, what I’m doing wrong? 🙂

(s/valid? (s/and string? (complement str/blank?)) “Hello”)
>Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval14632$fn$G (protocols.clj:11). No implementation of method: :conform* of protocol: #’clojure.alpha.spec.protocols/Spec found for class: clojure.lang.PersistentList

Alex Miller (Clojure team)12:03:26

You can’t use complement like that with spec 2 - it needs to be explicitly wrapped into a spec with s/spec

Alex Miller (Clojure team)12:03:51

Or you could use an anonymous function there too

pedrorgirardi12:03:51

Gotcha! Thank you, Alex.

Alex Miller (Clojure team)12:03:50

anonymous functions are explicitly handled as symbolic specs

pedrorgirardi14:03:28

Ah okay. I should have read the wiki with more attention. One other thing, does select sub-selections should work for nested unqualified keywords? The examples on the Wiki use qualified keywords, but then I try to nest sub-selections with unqualified keywords and it seems to only work for the “first level”.

pedrorgirardi14:03:04

Take this one for example

(s/select ::user [::id ::addr {::addr [::zip]}])
If we use an unqualified :addr and :zip, the spec doesn’t validate, or require, the`:zip` key.

Alex Miller (Clojure team)14:03:02

I think there is a bug in this area, someone else had a similar case

Alex Miller (Clojure team)14:03:17

there are several outstanding bugs in the schema/select stuff

pedrorgirardi14:03:42

Ah alright then. I was wondering what I was doing wrong.

Alex Miller (Clojure team)14:03:00

using unreleased software :)

pedrorgirardi14:03:28

Yep, schema and select are too nice 🙂

pedrorgirardi14:03:52

Well, thank you very much, Alex.

pedrorgirardi05:03:34

Whereas this works:

(s/valid? (s/and string? #(not (str/blank? %))) "Hello")

erwinrooijakkers08:03:44

Hi all, I generate a class like this:

(ns logging.logback-stackdriver-pattern-layout
  "Logback PatternLayout that logs in Stackdriver JSON format to stdout.
  It masks sensitive data with replacements from the pattern->replacement map."
  (:gen-class
   :extends ch.qos.logback.classic.PatternLayout
   :name logging.StackdriverPatternLayout)
  (:require
   [cheshire.core :as json]
   [clojure.string :as string])
  (:import
   (ch.qos.logback.classic Level)))
Then in the same project I have an XML logback-gcloud-appender.xml file that uses that class:
<appender name="GCLOUD" class="ch.qos.logback.core.ConsoleAppender">
  <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
    <layout class="logging.StackdriverPatternLayout"/>
  </encoder>
</appender>
Then from another project that uses the above project using lein-git-down. This project also has a logback.xml and refers to this logback-gcloud-appender.xml from the project’s resources:
<configuration scan="true" scanPeriod="10 seconds">
    <include resource="logback-gcloud-appender.xml"/>
    <root level="info">
        <appender-ref ref="GCLOUD"/>
    </root>
</configuration>
When running the project we see this error:
09:26:03,352 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [GCLOUD]
09:26:03,356 |-ERROR in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Could not create component [layout] of type [logging.StackdriverPatternLayout] java.lang.ClassNotFoundException: logging.StackdriverPatternLayout
	at java.lang.ClassNotFoundException: logging.StackdriverPatternLayout
	at 	at .URLClassLoader.findClass(URLClassLoader.java:382)
	at 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at 	at ch.qos.logback.core.util.Loader.loadClass(Loader.java:120)
	at 	at ch.qos.logback.core.joran.action.NestedComplexPropertyIA.begin(NestedComplexPropertyIA.java:102)
Why does the class logging.StackdriverPatternLayout lead to ClassNotFoundException ? This class should be generated?

p-himik08:03:58

Just to make sure - did you compile the CLJ code?

erwinrooijakkers08:03:09

I include it using lein-git-down

p-himik08:03:33

I have no idea what that means. Is the code that has :gen-class AOT'ed?

erwinrooijakkers08:03:40

Retrieving project-with-class/21e8b5bc0990eb5321694d38e2a676789426fd7a/project-with-class-21e8b5bc0990eb5321694d38e2a676789426fd7a.jar from private-bitbucket
Created /Users/erooijak/.gitlibs/libs/mediquest-nl/project-with-class/21e8b5bc0990eb5321694d38e2a676789426fd7a/target/project-with-class-0.0.0.jar

erwinrooijakkers08:03:30

Lein git down is a plugin that pulls code from a repository

p-himik08:03:39

The jar file that is ought to have the CLJ code - does it have the relevant class file?

erwinrooijakkers08:03:47

I don’t see it…

erwinrooijakkers08:03:04

It does have the clj file

p-himik08:03:05

The plugin then has nothing to do with it. You cannot just package the code that has :gen-class, you have to compile it. Ahead of time.

erwinrooijakkers08:03:37

I see how can that be enforced?

p-himik09:03:07

Likely Lein has some flag or plugin for it that you can use when you bundle the code. I have no idea - I don't use Lein at all.

erwinrooijakkers09:03:25

I see thanks for your suggestion I think that’s the way to go

👍 4
erwinrooijakkers09:03:19

This fixed it!

:aot [logging.logback-stackdriver-pattern-layout]

p-himik09:03:04

No problem. :)

jumar09:03:16

When working in a codebase, I was suprised that the REPL doesn't seem to pick redefinitions of certain functions; finally I've noticed that it's because they are defined in a hashmap like:

(def w {:fn other.ns/my-fn})

(defn wrapper []
  (-> w ...))

;; in other.ns
;; this function is changed and evaled but no changes detected!
(defn my-fn [] ...)
It was very confusing. Now if I reload the wrapper function / ns I can get new changes but it's not the best workflow and easy-to-forget. So I changed it to use #':
(def w {:fn #'other.ns/my-fn})
Would you say that that's a valid approach? Note: the wrapper fn isn't called in a tight loop

jumar09:03:30

I was aware of a similar issue with closures (e.g. when passing a ring handler to jetty) but this one stills surprised me. Any other contexts that this thing can manifest itself in?

dominicm09:03:24

I benchmarked the cost of vars, one sec

dominicm09:03:24

Hmm, not actually vars, but wrapper functions. Same idea though. Vars would be good to test.

p-himik10:03:00

It seems that the indirect use via lambdas ended up being a tiny bit faster. Do you have any idea why that might be the case?

jumar10:03:08

Thanks for sharing the numbers; for me it's not really a concern though 🙂. I'm mostly interested in knowing if this is the right thing to do (I assume it is) and then in what other situations it can happen.

dominicm10:03:08

@U2FRKM4TW I have no idea! That surprised me too, but I ran a few times to verify.

dominicm10:03:38

It is right, the other option is the lambda thing I did in my measurements

👍 4
dominicm10:03:09

The common response to your solution is "bad performance", which is why I measured :)

👍 4
Alex Miller (Clojure team)12:03:50

I’d use vars here personally

Alex Miller (Clojure team)12:03:20

They’re probably slower than lambdas due to the synchronization they incur

Alex Miller (Clojure team)13:03:04

although I guess if you're relying on a lambda call to the function to pick up changes, that has to go through the same synchronization, so not sure off the top of my head

dominicm13:03:09

Probably worth measuring ;)

Alex Miller (Clojure team)13:03:45

well unless it's a hot spot, I'd say it's probably not worth measuring... :)

dominicm13:03:03

I've had this debate over whether ring handlers should use this or not.

dominicm13:03:36

In some places it's hot. Most people are too lazy to measure.

dominicm13:03:31

It's handy to have numbers you can point at

Alex Miller (Clojure team)13:03:50

there used to be a significant perf issue in calling through vars for variadic functions (iirc) that was fixed in the last few years

Alex Miller (Clojure team)13:03:23

that was fixed in 1.9 - before that this case was noticeably slower

jumpnbrownweasel15:03:48

@alexmiller are you saying that every call through a defn synchronizes, on the var?

jumpnbrownweasel15:03:04

I must be misunderstanding.

Alex Miller (Clojure team)15:03:11

getting the value of a var means reading the root value, and access to that field is synchronized (so it can be safely set)

Alex Miller (Clojure team)15:03:05

or so, I should say it's a volatile read, not synchronized

jumpnbrownweasel15:03:27

ah, thanks, that's better

jumpnbrownweasel15:03:13

I see there is an option to avoid the volatile read as well, :direct-linking, which is nice.

Alex Miller (Clojure team)15:03:32

yes - you lose the reference then, but that's fine if you're compiling an app, not at the repl

jumpnbrownweasel15:03:48

I can imagine that hotspot may be able to do optimizations with :direct-linking that are not possible without it

jumpnbrownweasel15:03:35

such as inlining

Alex Miller (Clojure team)16:03:40

additionally, the compiled classes do not need to load and save the vars during class initialization, so it improves both startup time and execution time

Alex Miller (Clojure team)16:03:06

the clojure jar itself is compiled with direct linking

jumpnbrownweasel16:03:25

pretty cool that we can have such a dynamic system during dev, and still optimize in this way

colinkahn12:03:26

Are there any libraries that will give me all different paths of a tree? Like tree permutations?

benoit12:03:44

Without knowing too much about what you mean by "tree". You could zip your data structure, walk through it and return the path of each node. https://clojuredocs.org/clojure.zip/path

❤️ 4
colinkahn12:03:18

@U963A21SL I haven't used zip much, what would I use to walk it?

benoit12:03:28

If depth-first works for generating the paths, you can use clojure.zip/next.

colinkahn12:03:10

Yep, that'll work for my case

Ivar Refsdal11:03:28

I saw this video, The Art of Tree Shaping with zippers, after reading this thread: https://www.youtube.com/watch?v=5Nm56YvTKZY A good explanation IMHO. Slides here: http://arnebrasseur.net/talks/2018-clojure-zip-denver/#1 Based on those slides, I would find all-paths like this:

(def all-paths
  (->> my-zipper
       (iterate z/next)
       (take-while (complement z/end?))
       (map z/path)))
Works like a charm (without a recursive helper function).

colinkahn23:03:32

Playing off your snippet this is what I originally set out to do :

(defn full-path [loc]
  (conj (vec (zip/path loc)) (zip/node loc)))

(defn leaf-paths [z]
  (->> z
       (iterate zip/next)
       (take-while (complement zip/end?))
       (filter (complement zip/children))
       (map full-path)))

Ivar Refsdal07:03:53

Glad that it helped you! I'm hoping to (soon) use zippers for something that "starts" out at the leafs as well. Before I've typically recursed down a mix of map and vector structures, building up the result as I go. It has worked, but I find the code hard to reason about.

Ivar Refsdal07:03:18

(filter (complement zip/children)) ^^ Nice way of finding leafs by the way 🙂

Ramon Rios13:03:19

Hello folks. I have a doubt here. i have this vector:

[{:customer-id "cid-1"
          :customer "Customer 1"
          :policy "Policy 1"
          :term 1000
          :settlement 300
          :brio-nr 100}
         {:customer-id "cid-2"
          :customer "Customer 2"
          :policy "Policy 2"
          :term 1000
          :settlement 300
          :brio-nr 100}
         {:customer-id "cid-3"
          :customer "Customer 3"
          :policy "Policy 3"
          :term 1000
          :settlement 300
          :brio-nr 100}
         {:customer-id "cid-4"
          :customer "Customer 4"
          :policy "Policy 4"
          :term 1000
          :settlement 300
          :brio-nr 100}]
I would like to update a field in one of the maps inside of the vector. How could i do this? I was thinking to use merge but i look a the docs and saw that do not work with vectors 😞

noisesmith13:03:16

@ramonp.rios update-in can use vector index as part of its "path" argument

noisesmith13:03:40

user=> (update-in [{:a 1 :b {:c 3}} {:a 2 :b {:c 6}}] [0 :b :c] + 12)
[{:a 1, :b {:c 15}} {:a 2, :b {:c 6}}]

darwin13:03:30

also for more complex data queries/manipulation I would recommend learning specter[1] https://github.com/redplanetlabs/specter

teodorlu14:03:49

@ramonp.rios I'd consider just using map

Alex Miller (Clojure team)14:03:01

I would reconsider having a vector in the first place, vs indexing by customer-id

👆 8
Ramon Rios14:03:11

Thank you all of you! I'll try each of your suggestions

teodorlu14:03:58

Feel free to ask for clarification if you need to. Sometimes too many options makes it harder to pick one!

Ramon Rios18:03:54

Thanks. Your suggestion helped a lot

👍 4
dominicm15:03:02

At some point the problem with indexes is that you build a mini database.

dominicm15:03:19

You have different code which needs different indexes and it gets hard to manage :)

p-himik15:03:42

Kind of unavoidable, isn't it?

noisesmith15:03:30

clojure makes reindexing in new ways easy at least

dominicm15:03:33

Yeah. Then you end up with datascript, and that's it's own fun :).

dominicm15:03:41

Yeah, it isn't so bad. Not efficient though.

Alex Miller (Clojure team)15:03:56

or you can just use the clojure.set namespace, which implements the relational model :)

teodorlu15:03:54

@alexmiller Would you just stick maps next to each other then? Or would you do it on a per-property basis?

dpsutton15:03:00

it erased the gist link but this is @hiredman’s gist i always go back to at > https://gist.github.com/hiredman/7d17d8d2b58c41ce95bf2db305b0f427

8
teodorlu15:03:20

This is really interesting. Thanks for the reference.

borkdude17:03:38

if I want to have a core.async channel with a transducer on it, but the rest of the settings the same, what should I provide as buffer size, 0?

ghadi17:03:56

transducers need a buffer

hindol17:03:11

When using reduce with a string, does it optimize internally with a StringBuilder? Specifically, I want to know about this,

(defn reverse-string [s]
  (reduce (fn [coll elem] (str elem coll)) "" s))

ghadi17:03:21

no, that is O(n^2) in space

👍 4
ghadi17:03:35

you can use a custom reducing function

ghadi17:03:18

(defn into-str
  "reduce coll into a String, given a transducer"
  [xf coll]
  (transduce (comp xf (map str))
             (fn
               ([] (StringBuilder.))
               ([^StringBuilder sb] (.toString sb))
               ([^StringBuilder sb s] (.append sb ^String s)))
             coll))

👍 8
ghadi17:03:56

@hindol.adhya reduce is polymorphic on the source collection

ghadi17:03:23

it doesn't transform/understand the given reducing function

dpsutton17:03:08

i see that a lot on exercism submissions

hindol17:03:29

@dpsutton I just saw one. Wanted to confirm before advising "don't do this", 🙂

johnj17:03:34

to make clojure fast, write java

dpsutton17:03:17

yeah. i put some examples with time where i can do 50,000 loops in an order of magnitude less than their example can do in 500 loops.

Graham Seyffert21:03:33

@hindol.adhya str does use a StringBuilder internally, so you could probably just (apply str (reverse s))

16
hindol07:03:15

This I know. I was commenting on someone else's solution on exercism.