Fork me on GitHub
#beginners
<
2021-01-04
>
Ty07:01:05

My last little foray into clojure went really well. I have another idea I could use some validation for, as well as some direction on how to approach it. I watched a Pharo Smalltalk demo and thought the codebase visualization and reflection metrics were really cool. What I'd like to do is be able to parse a clojure codebase, run some metrics on that codebase, and then generate some visualization tools and maybe even a way to traverse the codebase. Some simple ideas might be showing the number/location of references to a function. Showing a hierarchy of macro usage, along with some example inputs/outputs that show macro expansion. In general being able to generate a graph of the codebase that spans out from the entry point(s). Would this type of tool be at all useful? Is there something like this already out there? If it would be useful, what are some additional things that also might be nice to have with a tool like this? Lastly, if I were to attempt this, what would be a good way to approach it? I'd really like to be able to fully parse the contents of a file/codebase so that I can have all the semantic information I need without having to resort to hacks/tricks.

jumar08:01:57

I had a similar idea/need a long time. There are some plugins/libs to visualize the call graph, e.g. clj-usage-graph: https://github.com/gfredericks/clj-usage-graph/blob/master/src/com/gfredericks/clj_usage_graph/usages.clj

Tim Robinson11:01:49

Can anyone explain why I would use force with a delay? We already have @ and deref and it seems to me they do exactly the same thing

noisesmith13:01:23

force is agnostic about whether its argument is actually a delay

user=> (force 42)
42

noisesmith13:01:19

and @ is already just a reader shorthand for deref

Tim Robinson17:01:12

ok thanks - I can't really think of a use case for that but I guess it was important to someone :)

roelof11:01:03

Why do I get here a error :

(defn to-json[suspects]
  into {} (clojure.string/join( ", " (get :name suspects) (get :glitter-index suspects))))


(to-json {:name "Roelof" :glitter-index 2})

roelof11:01:33

; Execution error (ClassCastException) at chapter4/to-json (form-init1060733672431633963.clj:82).
; class java.lang.String cannot be cast to class clojure.lang.IFn (java.lang.String is in module java.base of loader 'bootstrap'; clojure.lang.IFn is in unnamed module of loader 'app')

delaguardo11:01:35

because you wrote clojure.string/join(", " ...)

delaguardo11:01:24

to call a function in clojure you must use sexp form like this (clojure.string/join ", " ...)

roelof11:01:49

still no luck

roelof11:01:53

(defn to-json [suspects]
  (  into {} (clojure.string/join ", " [(get :name suspects)])))


(to-json {:name "Roelof" :glitter-index 2})

roelof12:01:04

; Execution error (IllegalArgumentException) at chapter4/to-json (form-init1060733672431633963.clj:82).
; Don't know how to create ISeq from: java.lang.Character

Antonio Bibiano12:01:54

get takes the map first and then the key

Antonio Bibiano12:01:14

(get suspects :name)

delaguardo12:01:59

join will return a string calling (into {} β€œsome string”) will throw that exception

delaguardo12:01:29

maybe you can describe what you trying to achieve?

Tim Robinson12:01:06

also [(get :name suspects)] is a vector with one entry, so string/join won't have anything to join

roelof12:01:11

what I try to achieve is to convert this : {:name "John", :glitterindex: 2) into a json without using a json library

roelof12:01:35

so the outcome is {"John Doe", 2}

dharrigan12:01:03

(def foo {:name "John Doe" :glitterindex 2})

dharrigan12:01:10

(apply hash-map (vals foo))

dharrigan12:01:14

{"John Doe" 2}

delaguardo12:01:13

maps in clojure is unordered so vals might return [2 "john"]

dharrigan12:01:39

However, here be dragons, json processing is very edge casey, lots of work arounds and cavets - hence a trivial translation won't suit

dharrigan12:01:06

I would just use a library tbh and @delaguardo is right πŸ™‚

roelof12:01:30

thanks, but I still think the purpose of the challenge of the brave book was to do it manually

delaguardo12:01:40

Then you can use juxt to extract values under certain keys from a map

roelof12:01:36

juxt ?? That one is not explained in thir or a former chapter

delaguardo12:01:00

could you share a link to your current chapter?

delaguardo12:01:26

otherwise I can only guess what you are trying to do)

delaguardo13:01:30

ok, so is it exercise 4 you are working on?

Antonio Bibiano13:01:38

Ah I just did that one

Antonio Bibiano13:01:52

if that's ok with you I can share my solution

Antonio Bibiano13:01:00

but I was converting to CSV

roelof13:01:18

yep, the lastest ine

roelof13:01:57

@antbbn of course you may. Maybe I find some ideas I can use

Antonio Bibiano15:01:21

(defn get-many
  [ks record]
  (map #(get record %) ks))
(defn suspects-to-csv 
  [suspects]
  (let [header-keys (keys (first suspects))]
    (reduce #(clojure.string/join "\n" [%1 %2]) 
            (clojure.string/join "," header-keys)
            (map 
             (comp (partial clojure.string/join ",") (partial get-many header-keys))
             suspects))))

Antonio Bibiano15:01:51

I used a let block to make sure that I always extract the keys in the same order

noisesmith15:01:54

get-many could just be (map record ks) if you are using normal hash maps

noisesmith15:01:59

also, keys and vals will always return the same order for two collections with the same keys

noisesmith15:01:08

finally your string/join calls could be str calls, if performance is a concern at all you could use a StringBuilder instead

Antonio Bibiano15:01:23

mhh I lost part of the conversation

noisesmith15:01:30

yeah, slack is being weird

Antonio Bibiano15:01:12

but now that I think about it totally makes sense that keys and vals will keep the order for the same collection

Antonio Bibiano17:01:33

The main reason why I used get-many (or (map record ks) as you suggested) is because I was not sure if vals will be in the same order for different maps with the same keys

Antonio Bibiano17:01:29

as far as I understand that is not guaranteed

clyfe17:01:57

Maps in clojure are not ordered. For ordered maps see https://github.com/flatland/ordered

clyfe17:01:58

PersistentArrayMap is, PersistentHashMap isn't, and past 8 pairs the former is "upgraded" to the latter

Antonio Bibiano17:01:55

so you think it's a good idea to make sure that you get the keys in the explicit order you want using something like (map record ks)

clyfe17:01:54

if you want order, use ordered linked above

seancorfield18:01:40

Although the order of keys (or vals) is "random", they are consistent: whatever order keys gives you matches whatever order vals gives you.

Antonio Bibiano18:01:08

gotcha, it's only that the gist of that exercise was to convert that list of hash maps to a csv

Antonio Bibiano18:01:49

and given that each map is a different element of the list even though they have the same keys

Antonio Bibiano18:01:28

I was worried about the case where vals of one item might return them in a different order than vals of another item

seancorfield18:01:10

Right, I think your overall approach -- get keys from the first hash map and use that, in order, to get values from the other maps makes sense. You can't rely on vals producing the same order of keys across multiple hash maps.

clyfe18:01:26

sorted-map or sorted-map-by can also be used to ensure a definite order on keys

Antonio Bibiano18:01:43

Nice! thanks for the tips

noisesmith18:01:10

@UCCHXTXV4 maps are not ordered, but to PersistentHashMap instances with the same keys will have a keys output and a vals output in the same order

clyfe18:01:58

PersistentArrayMap is ordered, PersistentHashMap isn't

noisesmith18:01:43

I didn't say it was orderd, I said vals or keys on two PersistentHashMap with the same keyset have the same order

πŸ‘ 3
noisesmith18:01:21

(ins)user=> (def m1 (into {} [[:a 0] [:b 1] [:c 2] [:d 3]]))
#'user/m1
(ins)user=> (def m2 (into {} [[:a "a"] [:b "b"] [:c "c"] [:d "d"]]))
#'user/m2
(ins)user=> (= (keys m1) (keys m2))
true

clyfe18:01:45

Yes, Sean said that too.

andy.fingerhut19:01:56

@U051SS2EU Not quite. keys and vals are only guaranteed to return orders consistent with each other for the same identical map object. While it is common for two PersistentHashMap objects that are = to return the same order as each other for keys (or vals), that is not guaranteed, and can be false when two keys have the same hash value, but were added to the not-identical-but-= PersistentHashMap instances.

πŸ’― 3
andy.fingerhut19:01:59

Example:

$ clj
Clojure 1.10.1
(clojure-version)
"1.10.1"
(hash 0)
0
(hash 92612215)
0
(def m1 (-> {} (assoc 0 :added-first) (assoc 92612215 :added-second)))
#'user/m1
(def m2 (-> {} (assoc 92612215 :added-first) (assoc 0 :added-second)))
#'user/m2
m1
{0 :added-first, 92612215 :added-second}
m2
{92612215 :added-first, 0 :added-second}
(= m1 m2)
false
(keys m1)
(0 92612215)
(keys m2)
(92612215 0)

πŸ‘ 6
andy.fingerhut19:01:00

This happens because PersistentHashMap puts all keys with the same hash value into the same "hash bucket", which is simply a linked list of elements, in the order that they were last added to the PersistentHashMap

noisesmith19:01:16

thanks for the info

jumar04:01:46

In Andy's example, the PersistentArrayMap is used so it preserves insertion order but the same holds for PersistentHashMap :

(def m1h (-> (hash-map :very-first -1) (assoc 0 :added-first) (assoc 92612215 :added-second)))
(keys m1h)
;; => (0 92612215 :very-first)

(def m2h (-> (hash-map :very-first -1) (assoc 92612215 :added-first) (assoc 0 :added-second)))
(keys m2h)
;; => (92612215 0 :very-first)

andy.fingerhut05:01:13

Ah, yes, sorry I didn't force the use of PersistentHashMap, but the example I had written up earlier did do so, as jumar's example does.

noisesmith15:01:52

(the main reason to use join instead of iterating str calls is performance, using small calls to join in a loop like this undoes that advantage)

noisesmith15:01:33

I guess the one gotcha is an ArrayMap vs. HashMap, which I didn't consider at first

popeye15:01:21

Hi, I have a map and i want to fetch the keys which has more that 10 value, How can I get it

{"hello" 10, "hi" 12}

popeye15:01:06

output should be {hi 12}

caumond16:01:06

Use filter

caumond16:01:34

(sorry, hi first !!)

seancorfield17:01:56

@popeyepwr (into {} (filter #(< 10 (val %)) coll)) (if the hint to "use filter" isn't enough on its own)

βœ… 6
😁 3
caumond19:01:18

Hey I did a full answer then, but making o e answer in multiple message was not a good idea with slack outage. The full answer never arrive !!

Antonio Bibiano17:01:33

The main reason why I used get-many (or (map record ks) as you suggested) is because I was not sure if vals will be in the same order for different maps with the same keys

Antonio Bibiano18:01:24

another thing that I noticed looking at my code is that I reduce over the result of a mapping over the list of suspects

Antonio Bibiano18:01:56

but It could also be refactored to reduce over the suspect by using a slightly more complex function

Antonio Bibiano18:01:03

(defn suspects-to-csv 
  [suspects]
  (let [header-keys (keys (first suspects))]
    (reduce (fn [result m]
              (str result 
                   "\n" 
                   (clojure.string/join "," (map m header-keys))))
            (clojure.string/join "," header-keys)
            suspects)))

Antonio Bibiano18:01:28

which style would be more idiomatic?

Max Deineko19:01:23

Reducing without building intermediate sequence first can be faster -- I'd think only noticeably so with larger sequences. As a beginner I usually find separate reduce and map steps sometimes easier to manipulate in code, but that's probably highly editor/user-dependent. I'd be surprised though if one of the two variants was less idiomatic than the other -- I'd expect readability/style preference to be criterium of choice here.

Antonio Bibiano20:01:02

Makes sense, also I agree that in this context and as a beginner reducing over something that is already mapped seems more straightforward

Kannan Ramamoorthy18:01:08

I’m fiddling to setup test files and CLI commands to run it using deps.edn. If anyone could point me out to any reference that would be helpful.

Kannan Ramamoorthy18:01:33

Is there any sample projects that I can look upto? I referred to @U04V70XH6’s https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn. And added below,

:test   {:extra-paths ["src/test/clojure"]
          :extra-deps  {org.clojure/test.check {:mvn/version "RELEASE"}}}

 :runner {:extra-deps {com.cognitect/test-runner
                       {:git/url ""
                        :sha     "b6b3193fcc42659d7e46ecd1884a228993441182"}}
          :main-opts  ["-m" "cognitect.test-runner"
                       "-d" "test"
                       "-d" "src/test/clojure"]}
but when I run clj -M:test or clj -M:runner , it takes me to repl instead of running the tests.

seancorfield18:01:25

@UMU908UNM Check clojure -Sdescribe -- I suspect you're on an old version.

seancorfield18:01:39

Most of my projects assume 1.10.1.727 or higher.

seancorfield18:01:40

For example https://github.com/seancorfield/clj-new and https://github.com/seancorfield/depstar both state 1.10.1.727 or higher as a requirement in their READMEs (my dot-clojure does not, but it probably should).

seancorfield18:01:22

All of my projects use deps.edn for testing these days -- https://github.com/seancorfield/next-jdbc/ is a good example since it has both GitHub Actions and CircleCI configured to run tests via deps.edn

Kannan Ramamoorthy18:01:34

My version is β€œ1.10.1.727”. But not sure why running clj -M:testΒ orΒ `clj -M:runner` takes me to repl. Will check some of your projects and compare my deps.edn. Thanks Sean!

pavlosmelissinos18:01:21

Aren't the :test and :runner aliases meant to be run together? like so: clj -M:test:runner Not sure about :runner on its own but the :test alias doesn't have a :main-opts entry, so I think it makes sense that it takes @UMU908UNM to a repl, no? :thinking_face:

seancorfield18:01:30

Oh, well spotted! Yes, you need -M:test:runner

seancorfield18:01:30

But clj -M:runner should not give you a REPL -- I would expect in most projects for it to give you an exception since you won't have any test dependencies added in.

πŸ‘ 3
seancorfield18:01:01

For example:

(! 511)-> clojure -X:new :name kannan/example
Downloading: seancorfield/clj-new/maven-metadata.xml from clojars
Generating a project called example based on the 'lib' template.
The lib template is intended for library projects, not applications.
(! 512)-> cd example/
(! 513)-> clojure -M:runner

Running tests in #{"test"}
Execution error (FileNotFoundException) at cognitect.test-runner/test (test_runner.clj:62).
Could not locate kannan/example_test__init.class, kannan/example_test.clj or kannan/example_test.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

Full report at:
/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/clojure-3814224161163105319.edn

pavlosmelissinos18:01:12

For what it's worth, here's what I'm getting in each case (this is for a project that doesn't have any tests yet):

> clj -M:test       
Clojure 1.10.1
user=> 
> clj -M:runner

Running tests in #{"test" "src/test/clojure"}

Testing user

Ran 0 tests containing 0 assertions.
0 failures, 0 errors.
> clj -M:test:runner

Running tests in #{"test" "src/test/clojure"}

Testing user

Ran 0 tests containing 0 assertions.
0 failures, 0 errors.
(I'm on v1.10.1.763)

seancorfield20:01:18

@UEQPKG7HQ Yup, it's going to depend on exactly what's in your project, with -M:test producing a REPL (since there's no :main-opts in that alias) and -M:runner either failing (as I showed above in a freshly created project) or finding no tests to run as you saw.

πŸ‘ 3
Kannan Ramamoorthy04:01:06

Got it working! Thank you Sean and Pavlos. Took a while to relaize that unlike Java, we have to have different namespace for test and actual source that we have to test. In java the package name stays the same for both the actual and test class.

Kannan Ramamoorthy04:01:55

I assume the reason being, ns is not exactly equivalent to package. But kind of equivalent to fully qualified class name.

seancorfield04:01:42

Ah, yes! The convention is that if you have src/foo/bar.clj (with (ns foo.bar ...)) then you would have test/foo/bar_test.clj (with (ns foo.bar-test ...)).

seancorfield04:01:09

If you create a brand new project (with clj-new) that's what you'll see...

seancorfield04:01:10

seanc@DESKTOP-30ICA76:~/clojure$ clojure -X:new :name foo.bar
Downloading: seancorfield/clj-new/maven-metadata.xml from clojars
Generating a project called foo.bar based on the 'lib' template.
The lib template is intended for library projects, not applications.
seanc@DESKTOP-30ICA76:~/clojure$ tree foo.bar
foo.bar
β”œβ”€β”€ CHANGELOG.md
β”œβ”€β”€ LICENSE
β”œβ”€β”€ README.md
β”œβ”€β”€ deps.edn
β”œβ”€β”€ doc
β”‚Β Β  └── intro.md
β”œβ”€β”€ pom.xml
β”œβ”€β”€ resources
β”œβ”€β”€ src
β”‚Β Β  └── foo
β”‚Β Β      └── bar.clj
└── test
    └── foo
        └── bar_test.clj

6 directories, 8 files
seanc@DESKTOP-30ICA76:~/clojure$ cd foo.bar/
seanc@DESKTOP-30ICA76:~/clojure/foo.bar$ clojure -M:test:runner

Running tests in #{"test"}

Testing foo.bar-test

FAIL in (a-test) (bar_test.clj:7)
FIXME, I fail.
expected: (= 0 1)
  actual: (not (= 0 1))

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
seanc@DESKTOP-30ICA76:~/clojure/foo.bar$ cat test/foo/bar_test.clj
(ns foo.bar-test
  (:require [clojure.test :refer :all]
            [foo.bar :refer :all]))

(deftest a-test
  (testing "FIXME, I fail."
    (is (= 0 1))))
seanc@DESKTOP-30ICA76:~/clojure/foo.bar$

seancorfield04:01:25

@UMU908UNM Hopefully that new project example above will be helpful!

πŸ‘ 3
Kannan Ramamoorthy05:01:47

Thanks a lot @U04V70XH6! That’s very informative!

Fra18:01:29

Hi, my question is probably very dumb and apologies for that. I’m trying to understand how this compojure-api code works (app (-> (mock/request :get "/api/documents?id=123")) works behind the scene. I printed in the repl both app and the mock request , you can find them here. https://paste.ofcode.org/gYrXDmzUzHTiYrQvKSzsxD what I don’t understand is how (app mock-request) is executed. I can’t find the function that receives the request. Also is not clear to me what #Route is printed just before any dictionary. Thanks for any help in advance.

clyfe18:01:12

See https://skillsmatter.com/skillscasts/3445-functional-web. Note: Compojure is old and slow O(n); works well with few routes, but generally prefer https://github.com/metosin/reitit.

πŸ™Œ 3
πŸ‘ 3
noisesmith18:01:04

@francesco.losciale #Route{} is a print format for a record called Route

noisesmith18:01:49

defrecord creates a type that acts like a hash map, but has its name prefixed when printing, and allows custom extension of protocol methods

πŸ™Œ 3
roelof18:01:32

@antbbn thanks for showing me this

πŸ™Œ 3
Daniel Tobias20:01:07

hi all, im using cljs with reagent which produces an html and a js. My backend webserver is trying to render the page with jinja2 and i wanted to load the page with some data (it's a json object or an array in practice). How can i 'access' this data in cljs?

noisesmith21:01:08

you can use inline js to embed the data in the page, and then access the data with cljs interop

noisesmith21:01:31

more commonly the rendering happens in cljs and the data is fetched via AJAX from the server

Daniel Tobias23:01:36

I guess i could do ajax here but im changing the design from being written in straight js to cljs and thats how it currently works. I'll look into ajax though, thanks