Fork me on GitHub
#beginners
<
2022-06-01
>
tbtb08:06:44

This is probably a really simple problem to have but I can't figure out a good way to solve it. I've got two vectors of maps that have a common key and I'd like to group them by that key such that:

(def vec-1 [
{:my-key "bob" :my-val 1}
{:my-key "bob" :my-val 2}
{:my-key "fred" :my-val 3}])
(def vec-2 [
{:my-key "bob" :other-val 4}
{:my-key "bob" :other-val 5}])
Becomes:
{"bob" {
:vec-1 
[{:my-key "bob" :my-val 1}
{:my-key "bob" :my-val 2}]
:vec-2
[{:my-key "bob" :other-val 4}
{:my-key "bob" :other-val 5}]}
"fred" {
:vec-1 [{:my-key "fred" :my-val 3}]}}
I'm assuming it's some combination of group-by, map, etc, but I don't think I've grokked clojure enough in my head to work it out.

Bart Kleijngeld09:06:59

Is it always two vectors? Or could it be more, say, coming from a sequence? (This may not matter, since you can probably reduce it, but just making sure)

tbtb09:06:43

It is actually two sequences. I just said vectors to make the example code easier.

👍 1
tbtb09:06:48

I did think I was being clever using a merge-with to assign each to a key-value-pair, but then it fell over when a value only existed in one collection 😞

Bart Kleijngeld09:06:51

merge-with seems handy here. It's the :vec-1 and :vec-2 keys that I'm struggling with to get in there. (btw I'm a beginner, just trying to help and learn!)

Bart Kleijngeld09:06:47

(merge-with merge
  (group-by :my-key vec-1)
  (group-by :my-key vec-2))

;; yields:

{"bob" [{:my-key "bob", :my-val 1} {:my-key "bob", :my-val 2} [{:my-key "bob", :other-val 4} {:my-key "bob", :other-val 5}]], "fred" [{:my-key "fred", :my-val 3}]}
But that's quite far from the goal still 😅

tbtb09:06:53

Yeah, I thought I was being clever by doing something like:

(merge-with (fn [fir sec] {
:vec-1 fir
:vec-2 sec})
(group-by :my-key vec-1)
(group-by :my-key vec-2))
But if a my-key only existed in one collection, it just got thrown in there without my clever assignment 😞

tbtb09:06:15

I appreciate the help! Every step along the journey gets me closer 😄

Bart Kleijngeld09:06:18

Nice one! Shame it doesn't quite work. If I come up with something I'll let you know. Meanwhile I'm interested in what you and others will try and find

tbtb09:06:38

Thanks for your help along the way 🙂

Bart Kleijngeld09:06:49

If one doesn't contain :my-key, group-by groups it with key nil. So, I think you should be able to filter the nil keys out of the group-by result, before merging them.

Bart Kleijngeld09:06:08

(merge-with (fn [fir sec] {
:vec-1 fir
:vec-2 sec})
(into {} (filter #(some? (first %)) (group-by :my-key vec-1)))
(into {} (filter #(some? (first %)) (group-by :my-key vec-2))))
I think this works (needs a lot of cleanup and reusing, but gotta start by making it work)

tbtb09:06:10

Sadly, that one has the same issue that if I do (get <that-code> "fred") fred has been merged in without the :vec-1 :vec-2 thing.

plexus09:06:57

here's my attempt

(def input
  {:vec-1 [{:my-key "bob" :my-val 1}
           {:my-key "bob" :my-val 2}
           {:my-key "fred" :my-val 3}]
   :vec-2 [{:my-key "bob" :other-val 4}
           {:my-key "bob" :other-val 5}]})

(reduce
 (fn [acc [kw m]]
   (update-in acc [(:my-key m) kw] (fnil conj []) m))
 {}
 (for [[kw ms] input
       m ms]
   [kw m]))

plexus09:06:28

first use a for loop to "flatten" the input, then use update-in to put each entry in its correct spot

tbtb09:06:46

I got something that I think works:

(let [ls-1 (update-vals (group-by :my-key vec-1) #(hash-map :vec-1 %))
ls-2 (update-vals (group-by :my-key vec-2) #(hash-map :vec-2 %))]
(merge-with merge ls-1 ls-2))
Thanks to @U03BYUDJLJF for the merge-with merge

❤️ 1
plexus09:06:50

I'm sure folks can come up with more elegant solutions. There are two parts that make this a non-trivial question, the two levels of nesting in the result map, and the mutliple maps in the input. The double nesting makes it hard to come up with something general using e.g. group-by.

tbtb09:06:02

But I'd like to try the reduce version

plexus09:06:36

you can do almost anything with reduce so it's a good thing to be comfortable with, but most things can be done more easily with more specific functions

tbtb09:06:05

The reduce version seems to do just what I need too! Thanks! I also never saw fnil before, so learned something generic to go with the more specific 🙂

tbtb09:06:56

Thanks to @U03BYUDJLJF and @U07FP7QJ0 :thumbsup: I would've been banging my head on that for a long long time 🙂

yuhan10:06:01

You could also go with nested reduces:

(reduce-kv
  (fn [acc k ms]
    (reduce
      (fn [acc' m]
        (update-in acc' [(:my-key m) k] (fnil conj []) m))
      acc vs))
  {} input)
But I think the technique of 'normalising' the data before operating on it makes it easier to understand

Ed11:06:47

(let [vec-1 [{:my-key "bob" :my-val 1}
               {:my-key "bob" :my-val 2}
               {:my-key "fred" :my-val 3}]
        vec-2 [{:my-key "bob" :other-val 4}
               {:my-key "bob" :other-val 5}]]
    (transduce (mapcat (fn [[k vs]] (map #(vector k %) vs)))
               (completing (fn [m [k {:keys [my-key] :as v}]] (update-in m [k my-key] (fnil conj []) v)))
               {}
               [[:vec-1 vec-1] [:vec-2 vec-2]]))
this is my attempt. I've tried to tag each element as vec-1 or vec-2 and then used the reducing function to do the nested group-by.

Stef Coetzee10:06:47

Spun my wheels a little yesterday because I did not add "resources" to :paths in deps.edn when trying to serve static resources. Sharing so that it might save someone some time. Specify it like so:

;; deps.edn

{:paths ["src" "resources"]
 
 :deps {org.clojure/clojure {:mvn/version "1.11.1"}
        ring/ring {:mvn/version "1.9.5"}
        compojure/compojure {:mvn/version "1.7.0"}}}
In this instance, I was following along with a tutorial that makes use of Leiningen. The resources directory is included by default in Leiningen. Using deps.edn, this path has to be specified explicitly, just like src is. For more, see https://clojure.org/reference/deps_and_cli#_paths. You'll find that the route to some specified static resource (e.g. ) isn't found (returns a 404 error), even though you might have helpfully specified the directory within resources like so:
;; .../core.clj

...

(defroutes routes
  (GET "/" [] (resource-response "public/index.html")))

(def app
  (-> routes
      (wrap-resource "public")))

...
In this example, the CSS file resides in /resources/public/css. Happy static resource serving! 😁

👏 2
2
👍 1
Ted Ciafardini12:06:00

Question about something I’ve seen in project.clj files I’m not getting much out of google searches for -Dconf as in: :jvm-opts ["-Dconf=dev-config.edn" ] Would appreciate any resources/explanations of what’s going on here. I see that it’s targeting a specific edn config depending on the profile - but would love to have a deeper understanding

Alex Miller (Clojure team)13:06:01

-D is a jvm thing to set a Java system property (which can be read in your program via (System/getProperty “conf”)

gratitude 1
Ted Ciafardini13:06:56

Ah, I see. Thanks.

sheluchin19:06:54

I too have tried to Google for this without much luck. It seems like it's assumed knowledge :man-shrugging:

Bart Kleijngeld14:06:38

(if (> max-count 1) :max max-count)
Can I write this more concise without having to repeat max-count? Something that says "if some property regarding max-count is satisfied, then evaluate success body, and otherwise evaluate max-count" General advice is welcome, I'm fine having to repeat it, I think it might be unavoidable.

Matthew Curry15:06:47

Well if it's a pattern you see a lot, that is a job for a macro

tbtb15:06:02

(defmacro if-ret-or
  [p value else-val]
  `(if (~p ~value) ~value ~else-val))

(let [max-count 3]
  (if-ret-or #(> % 1) max-count :max))
Is probably not the cleanest way in the world to do it. My macro-fu is weak, but does work

👍 2
Matthew Curry15:06:55

Actually that could just be a function, seeing that written out

tbtb15:06:22

I guess the macro would be better for situations like

(let [max-count 3]
  (if-ret-or #(> % 1) max-count (do (Thread/sleep 3000) :max)))

tbtb15:06:55

Even though that doesn't need to go into the sleep branch, it will always take 3000 ms if a function rather than a macro

tbtb15:06:05

... I think.

Matthew Curry15:06:26

sure, arguments are unevaluated in a macro, but still better to prefer functions: (defn clamp [test thing limit] (if (test thing) limit thing))

didibus16:06:49

I don't think you can get much shorter, because it isn't clear what in the condition you'd want to return otherwise, like why isn't it:

(if (> max-count 1) :max 1)
For example, or imagine:
(if (> max-count counted) :max counted)
Which means you'll have to repeat max-count somehow to indicate that this is the thing to return in the else branch from the condition.

seancorfield19:06:08

@U0K064KQV You mean if there, not when, right? the code has been corrected

didibus21:06:17

Oups, yes, corrected

Drew Verlee23:06:08

Bart, your not repeating yourself there, there is no natural relationship between a keyword like :max and a number. If there was, in guessing you could derive the result another way

seancorfield23:06:36

@U0DJ4T5U1 He's asking about the max-count variable there -- in a sort of general (if (pred x) x y) kind of thing, where x has to be repeated.

Drew Verlee23:06:48

I understand, the technical answer is that a macro could do that. probably just a function to... However i'm saying such a macro wouldn't remove the repetition from the readers mind because they would have to unwrap it to the more extended version as it's not common place enough to probably have hot loaded in memory.

👍 1
Drew Verlee23:06:22

another way to say it is that (min max-count some-number) would avoid repeating max-count and return max-count when it was the min. But it returns some-number in the other cases.

Drew Verlee00:06:17

hopefully that makes sense. This is what i was in code form

(def max-count 2)
(def result (if (> max-count 1) :max max-count))
;; => :max

(if (= result :max)
  :user-is-a-super-star
  :user-is-just-average)
;; => :user-is-a-super-star


(def result (min max-count 3))
;; => 2

;; somewhere else in code
(if (= result max-count)
  :user-is-a-super-star
  :user-is-just-average)
;; => :user-is-a-super-star

Bart Kleijngeld06:06:43

Thanks everyone. This is very useful and learnful. @U0K064KQV, you made me realize I perhaps haven't thought it through well enough. @U03HVDTLB9S's macro however seems to be what I'm looking for, nice work!

Léo Crapart18:06:43

Hi i'm new to this slack, (but have one year free time experience with clojure) and i'd like to know how junior programmer can start in professional jobs. I've seen some jobs offers and it looks like every company asks for 2+ years of xp in clojure OR many years of xp in software development (like senior java dev). My question is, how can I get started as a junior ? (professionnaly wise, not looking for learning material here) Thank's in advance :)

V20:06:05

You can apply to all those positions, even if you lack he experience. What you have to do is to have something that shows your skills with Clojure. That can be hobby projects, contributing to community projects at what not. If you have something to show, it is more likely that companies will take you in with less experience than required Edit; It is of course important to also have the knowledge of the domain you are applying for. If you apply for a backend position, then you might want to build some reitit/pedestal backend application of your own in clojure

❤️ 1
plexus20:06:15

Hi Léo, the answer as so often is... "it depends". It's true that there aren't a lot of junior Clojure jobs, it's been often the case in the past that people learn Clojure after they have some experience in another language, and so companies have had the luxury that they didn't really have to look for juniors. There also just weren't (and to an extent still aren't) that many people who learn Clojure as a first language and are then looking for jobs. That said companies do exist that will hire more junior people and invest in their learning. Do you have any other experience that might be relevant? Do you know HTML/CSS? Have you done any courses or workshops? Are you a good communicator? Any personal projects you can show?

practicalli-johnny20:06:54

Starting any career can be a huge challenge and its usually a combination of luck and determination, a good helping of enthusiasm to learn usually helps too (especially if you do not have experience of the business domain or technical experiences to draw from) Understanding the basic tools and practices of software development - git, basics of continuous integration (e.g. a few GitHub actions), writing unit tests, maybe a bit of Amazon Web Services or similar platform. This will make it easier for companies to hire you, as they wont feel they need to train you in this stuff before you do anything. Its the major complaint companies have with Universities that students don't have these basic skills to deliver code. Example projects can be useful (and putting them on GitHub or GitLab shows you can use the basics of Git). Projects dont have to be perfect and can focus more on what you are learning, including comments on why certain design decisions. You could also take someone elses project or tutorial and try different things and say what you liked about a different approach or why the new approach turned out to be not so good. An example project can be useful to talk about in an interview and discuss those choices, what you weren't certain about, what you think may be done better in a different way, etc I know quite a few people who have got their first coding jobs using Clojure, although it is far more common that companies look for seniors. There are many more jobs available when not limited to Cljoure (although they may or may not be as enjoyable) and those jobs may have some scope to introduce the language, especially on internal or tooling projects along-side other work. Keep applying to jobs, even though it can be very frustrating. Its actually frustrating almost everyone too and most companies are pretty terrible at hiring & interviewing, sorry. Remember that if a company cannot see the value of hiring you after you have told them why they should, then its a failing of that company to recognise your talents (and they've probably dont that with many other people too). Try learn what you can improve from each job application (and interview if you get them), although unfortunately feedback from the company is rarely forthcoming. Dont be afraid to ask the person interviewing you if they can think of any reason they shouldn't hire you (and try take their comments constructively - if they do say anything)

👍 1
Léo Crapart07:06:39

ok alright i'll continue to apply for jobs and do more personal projects then.

💪 1
Bhougland20:06:31

Has there been any work on a repl that support the "Debug Adapter Protocol"? https://microsoft.github.io/debug-adapter-protocol/

seancorfield20:06:26

We discourage cross-posting the same question into multiple channels so I deleted the duplicate of this in #tools-build (where it was off-topic anyway). I'm not sure where the best place to ask this question really is. I suspect the closest sort of work to this is in #cider which provides debugging facilities for Clojure via nREPL connections...?

Bhougland20:06:37

Makes sense, thanks Sean

practicalli-johnny20:06:32

There is some work on a Clojure DAP mode for Emacs, although I dont know how mature as I havent tried it yet and the docs seem very sparse and JVM centric https://emacs-lsp.github.io/dap-mode/page/clojure/ The ClojureScript docs seem to be more detailed, although I know someone who is still struggling to get this working as some of the configuratin isnt working https://emacs-lsp.github.io/lsp-mode/tutorials/debugging-clojure-script/ So yes there has definately been DAP work done, although I suspect it needs some volunteers to help it along for wider use.

practicalli-johnny20:06:10

I havent felt the need for a debugger in Clojure very much, although the cider-debug tool is very nice way of stepping through the algorithm of one or more functions https://practical.li/spacemacs/debug-clojure/

practicalli-johnny20:06:04

Thee is also the Sayid debugger, although that requires a little bit more investment to make use of than the cider-debug tool. I believe Sayid could be used with any Editor / tools, however it does come with an Emacs Cider plugin, so that seems to be the easier path to use. https://github.com/clojure-emacs/sayid

seancorfield21:06:41

I've found tap> (and Portal) to be very powerful for any debugging I've needed to do. I honestly can't say I've found breakpoint/step debugging useful in any language since I stopped doing Java stuff about fifteen years ago (and C++ before that, where step debugging was definitely useful!).

seancorfield21:06:27

The more dynamic and more interactive the development process, the less I have found the need for that sort of thing. Still, quite a few people really seem to like that workflow...

practicalli-johnny07:06:34

I found a breakpoint debug tool (e.g cider-debug) invaluable when learning Clojure, especially when writing some (overly complex) loop-recur code that I struggled to work out why I was getting the wrong result. However, I concur with Sean that once comfortable with Clojure then there is far greater use of data inspectors and a workflow that evaluates specific pieces of code to understand what is happening. This approach also encouraged me to write simpler functions, leading to far simpler code to evolve and maintain

Bhougland12:06:21

Thanks for the information. I know the name says debugger and it is definitely used for that in other language, but it is more the "standard" way of connecting a repl that I think could be beneficial. I am trying out the helix editor (like vim but with kakoune editing style with tree-sitter built in) and the clojure-lsp plugged in, clojure tree-sitter will too, but connecting a repl is more work unless it uses this protocol.

practicalli-johnny13:06:34

In terms of Clojure REPL connection implementation, DAP seems to be a decade behind. nREPL is the defacto standard used by almost all Clojure aware editors. For example, the developer of Conjure for Vim/Neovim considered Socket REPL and PREPL, before adopting nREPL as the main approach. Socket REPL does have an advantage of being built in to the Clojure standard library and Prepl is an interesting approach. So there are several Clojure specific protocol standards (some others not mentioned too) It will be interesting to see if the same amount of effort (a seriously huge amount) that has been put into Clojure LSP is also put into Clojure DAP support (or Socket REPL) Until then, I assume most tooling will adopt nREPL or Socket REPL (or both). I have no idea how any of these compare to DAP on a technical level, although I assume there must be a lot of commonality at least in concepts.

seancorfield17:06:22

@UT2EHQN7Q We run socket REPLs in several production processes because all you need is a JVM option at startup -- no dependencies, it's all built-in to Clojure's core -- and you get a standard plain-text REPL with full Clojure power. Between that and all the work invested in nREPL by all the different editor plugin teams (and the nREPL/CIDER folks themselves), I can't imagine DAP taking off in the Clojure world. About the only value that I see in it is a "standard" way to attach to a running process that wasn't started with that JVM option and without nREPL as a dependency. Part of the issue is that DAP is offering a specific and in Clojure terms very limited API, whereas a REPL running in a process gives you access to Clojure's full dynamic power at runtime. Perhaps someone will figure out how to use DAP to "sideload" nREPL and start an nREPL server in a process that otherwise didn't have it and doesn't have a socket REPL running?