Fork me on GitHub
#beginners
<
2020-07-16
>
mcgriff.benjamin05:07:40

@seancorfield I can't thank you enough for your CLI+deps.edn video. I was looking for a hands-on "ground-up" approach to setting up my projects after dealing with frustrating failures working through tutorials with lein. I now have a comprehensive understanding to how clj handles dependencies and aliases. With it being bare-bones, I'm also sort of forced to read through library documentation to plan out how I'm going to use it. One major thing I'm unsure of now is how build orchestration is implemented... Judging from the discussion in https://clojureverse.org/t/is-there-a-sales-pitch-for-switching-to-deps-edn-from-lein-in-2020/5367/56 though I think I am up against a long learning journey (like what are uberjars? Should I care about auto-reloading? Why are there so many overheads to setting up Clojure?). So much more to say but I don't want to meander away from this thank you post 😁 . Cheers

seancorfield05:07:55

@mcgriff.benjamin Glad you found it useful and, yes, Clojure is a long learning journey no matter what tooling you use I think. I do think "simple" composable tooling is easier to reason about tho'.

seancorfield05:07:50

Personally, I advocate avoiding the reload/refresh stuff and instead focusing on a tight eval-every-change REPL workflow (again, simple rather than easy).

e74905:07:37

what's the most popular/simplest approach to adding static typing to clojure? I looked at typed.clojure but also see some posts saying people aren't using it anymore/too overbearing

seancorfield05:07:32

@e749 Clojure is inherently a dynamically typed language. There are no static type systems for it.

seancorfield05:07:25

Typed Clojure is an interesting academic exercise but it is neither popular nor simple (I have used it -- and abandoned it -- twice).

e74905:07:00

@seancorfield (hah, hi! i was just reading your posts) yeah, i guess i'm trying to ascertain if it's usable IRL and it seems answerr is non

noisesmith05:07:17

not only that, but the vm itself is hostile to static typing - it allows extension of existing classes to new instances that are (supposedly, but not checkably) drop in replacements

seancorfield05:07:29

We found it unusable in real world code, yes.

e74905:07:35

gotcha; maybe some meta questions is how do you approach refactoring? I find myself wrestling with errors that really should be caught at compile time

seancorfield05:07:51

A good example of just how dynamic Clojure's type system is: protocols can be satisfied on a per-value basis using metadata (if they are declared to be open for that).

noisesmith05:07:53

that's the thing I really missed coming from OCaml to clojure

noisesmith05:07:06

it requires a different sort of discipline about how we define things

e74905:07:09

(context: coming from C and i realize this is probably not common to clojure, but i like to push dynamic typing to the edges of the system, much like mutability)

seancorfield05:07:34

@e749 In some ways Clojure pushes static typing to the edges.

noisesmith05:07:37

you honestly can't really push runtime typing it to the edge here, it's in the bytecode level

e74905:07:18

yeah makes sense; i got my hopes up that seeing the typed clojure project

seancorfield05:07:20

Can you be a bit more specific about the "wrestling with errors" that you are encountering?

noisesmith05:07:36

you can use / leverage static promises (see spec, and some optimizations or interop that needs type hints)

e74905:07:54

sure (also i haven't gotten to reading about clojure.spec which i had mis-thought was a testing framework)

noisesmith05:07:21

@e749 to see where this goes really wrong, look at scala, where they have an utter mess of a type system, the root cause of which is trying to impose ML semantics on a dynamically typed runtime

seancorfield05:07:41

@e749 Here's a post about how we use Spec at work https://corfield.org/blog/2019/09/13/using-spec/

seancorfield05:07:11

We've been heavy users of it in dev, test, and production since the early alphas of Clojure 1.9.

e74905:07:39

errors i'm annoyed at: I rename a member variable in "a constructor function" => forget to appropriately update it everywhere else I add a member variable to a defrecord => forget to update everywhere it gets created with new required data

galtelino17:07:43

what IDE are you using, if any? you can find out changed function names easily with Cursive, for example.

e74916:07:53

vscode + calva

noisesmith05:07:47

@e749 to be fair, spec is closer to a testing framework than it is a type system but we can get some of the nice features of both from it

e74905:07:19

@seancorfield thanks! reading post now @noisesmith can you elaborate?

noisesmith05:07:24

@e749 right, the compiler doesn't and won't help you with those things in clojure, the cost of entry is refactoring is no longer easy

noisesmith05:07:19

@e749 spec gives you the ability to make declarations about data, but doesn't actually check it until you explicitly make it do so, and can't do the checks statically

seancorfield05:07:34

I'm not sure what you mean by "member variable" and "constructor function" -- sounds like OOP to me...

noisesmith05:07:13

@seancorfield but we have the same problem when we rename keywords in hash-maps

e74905:07:57

@seancorfield just functions that return a hashmap of keyvals (i heard someone on a clojure talk refer to them as constructor functions)

seancorfield05:07:58

I see people talking about that but I can't say that I've found it a problem (in a decade of production Clojure usage).

seancorfield05:07:42

I guess some people are building lots of code before thinking about their attribute names and descriptions?

noisesmith05:07:55

@seancorfield when I wrote OCaml code, I could redesign my algorithm, and update the data structures I used, and the compiler would simply give me a list of the changes that would require from helper code and client code

noisesmith05:07:10

in clojure, I need to do this by hand (and do it much less often)

e74905:07:15

@seancorfield gotcha; maybe a higher level approach question then - you have a new senior engineer joining your team from C/C++; what would you tell them (or books/resources) on how to approach coding in clojure?

e74905:07:25

yeah, i'm not running into problems right now per say since my codebase is small but i get nervous at every change (for context, i'm writing a dsl compiler and iterating on the ast/high level IR/low level IR)

noisesmith05:07:38

you tagged Sean but I'll answer with two books: Joy of Clojure and Elements of Clojure - both lean heavily on the architecture / refactorability side of things more than other books do

e74905:07:14

@noisesmith ah perfect, thanks!

seancorfield05:07:18

(yeah, I'll agree with those... sorry, wife's going to bed and I have a cat on my lap so it's harder to respond right now than usual!)

e74905:07:42

no worries and very much appreciate the responses

noisesmith05:07:07

also both are IMHO instances of the rare books about programming in $LANGUAGE that remain useful and insightful if you never use $LANGUAGE again

seancorfield05:07:19

I think Clojure Applied is a good read -- although consider that the author says he would use records a lot less if he were writing that book again today.

noisesmith05:07:30

@e749 more concretely (and not directly addressing your questions so far) - this is the go to post about when and how to use custom types https://cemerick.com/2011/07/05/flowchart-for-choosing-the-right-clojure-type-definition-form/ - it's opinionated but covers a lot of community standards

e74905:07:39

yeah, i've been finding the most "emotionally frustrating" part is finding the rare resource that targets "new to clojure, not new to programming" niche

seancorfield05:07:55

@e749 I'm not sure I'd hire a senior C/C++ person onto my Clojure team, to be honest. And I say that as someone who helped design C++ (I was a member of ANSI X3J16 for eight years and secretary of the committee for three of those years).

noisesmith05:07:03

in practice, that flow chart above leads you to "use a hash map" ~98% of the time

e74905:07:21

lol, please do (withheld diatrabe on how fedup i am with C++ and it needs to be starved to death) :P

seancorfield05:07:21

It would depend on what else was in their background but "traditional OOP" is probably the hardest shift to make when coming to Clojure.

e74905:07:04

yeah definitely, fwiw, the reason i finally decided to bite the productivity bullet was I got tired of reimplementing ideas from clojure in C++

seancorfield05:07:51

(ah, finally... the cat has moved! now I can type comfortably!)

e74905:07:12

Joy Of Clojure looks like exactly what i was looking for (higher level, discussion of what's idiomatic, and why)

seancorfield05:07:08

C++ is good for what it was designed for, but both it and Java tend to teach a lot of habits/thinking that sort of needs to be unlearned in order to really get into Clojure and FP and become idiomatic about your solutions.

e74905:07:22

do you have talks (or know of) that center more around practical lessons learned usingn clojure in production at scale?

e74905:07:00

*clarification: lessons meaning problems encountered in real world surrounding programming e.g. 1 person clojure project vs. 10 people vs. 100 LOC of 100s lines vs. 100ks vs 1ms

e74905:07:13

or phrased a different way, the category of lessons you would tell yourself 10 years ago

seancorfield05:07:48

Also perhaps some of the domain driven design with clojure stuff (Google/Bing should find those easily enough).

seancorfield05:07:21

Our team has been 2-3 engineers across that decade, and we're at around 100,000 lines of code now.

e74905:07:47

:+1: thanks!

e74905:07:14

ah one last quick meta question: over time, are clojure talks generally {still hold true, hit or miss, probably not longer valid}? (for example, my C++ version of this to myself: "community at large is trend driven, cross-check old talks to verify the guidance hasn't flipped 180 or been deprecated")

noisesmith05:07:03

clojure is much more stable, though there are certain features the implementers had hopes for that will still be available but not encouraged (eg. reducers)

seancorfield05:07:00

A lot of Clojure talks are fairly timeless, but there are a few exceptions. Usually, the more specific they are about a particular library, the more they should be reviewed for more up-to-date alternatives. But pretty much everything by Stu Halloway and Rich Hickey are still solid 🙂

noisesmith05:07:03

you do need to watch out for people over-using things that are new (eg. so many bad uses of core.async came out in the first few years it existed...)

seancorfield05:07:34

Ah, yeah, Rich's early talks on reducers have been superseded by his talks on transducers. But that's a rarity amongst his output.

e74905:07:20

:+1: yeah, i've loved Rich's talks over the years as an outsider

e74905:07:51

is there any big "oops/here be dragons" or "stay away from this?" (bought joy of clojure but asking since it was released in 2014)

seancorfield05:07:57

I go back and read the transcripts of his talks over and over, and often get something new out of them each reading.

seancorfield06:07:13

Is that the 2nd Ed of JoC?

e74906:07:36

i believe so; the other version was from 2011

seancorfield06:07:15

Wow, time flies. JoC 1st Ed was my first Clojure book I think, either that or Clojure in Action. Those were def. my first two.

noisesmith06:07:32

no features described in JoC are deprecated IIRC - it's pretty timeless

seancorfield06:07:08

This gives a good sense of what has changed in each Clojure version -- you'll see it's fairly conservative https://github.com/clojure/clojure/blob/master/changes.md

seancorfield06:07:56

(and this is a quick reference for the timeline https://en.wikipedia.org/wiki/Clojure#Language_overview )

seancorfield06:07:52

At work, we've jumped on the alphas of every release since 1.3 and taken them to production -- they are very stable, in general, and the feature set is almost entirely additive across that whole decade.

e74906:07:46

brilliant; really appreciate it guys. hard to scour the internet for this type of "tribal knowledge"

seancorfield06:07:37

I'd say "avoid pmap" -- it's almost never what you want. Embrace interop -- the Java standard library is battle-tested and great for production use. You probably will never need ref's -- atom's are nearly always enough.

seancorfield06:07:31

Oh, and avoid macros unless you really need their features (lack of evaluating arguments, syntax enhancements) -- prefer functions because they're more composable. Also, avoid named arguments (trailing & {:keys [...]} stuff -- pass an options hash map instead.

seancorfield06:07:23

Good advice here https://stuartsierra.com/tag/dos-and-donts (from the Clojure in the Large speaker).

seancorfield06:07:38

And with that, I'm off for the night.

e74906:07:39

:+1: btw, if any other tips or books/resources in that vein pop into your head, I'm all ears I'm compiling this in my notes as a cheatsheet and will post it up

seancorfield06:07:36

Do you know about http://clojurians.zulipchat.com as a place to search the archives of this Slack? (since it's a free plan here and only 10,000 messages remain searchable -- but most channels are archived to Zulip and can easily be searched there)

e74906:07:08

haha nope! was just lamenting that this is a slack and info is lost best nugget of info to end on. cheers

patrick.farwick05:07:24

Sorry to break up the conversation with a quick question: Is there an easy way to read the first line of a BufferedInputStream in a similar way you can with .readLine when using LineNumberingPushbackReader?

noisesmith05:07:12

@patrick.farwick why not wrap the BufferedInputStream in a LineNumberingPushbackReader?

patrick.farwick05:07:05

That would work. Just didnt really realize I could do that. I'll look into it now lol

noisesmith05:07:37

there needs to be two layers of conversion, iirc, helps here

noisesmith05:07:17

(-> buffered (io/reader) (LineNumberingPushbackReader.)) I think

patrick.farwick05:07:37

that worked perfectly

patrick.farwick05:07:42

Super appreciate the help 🙂

noisesmith05:07:03

since this is #beginners - usually my approach when I need to use some class X is look at the data types I have, then start looking for constructors or static conversion functions that connect the dots

noisesmith05:07:36

it's easy with LNPR because it takes a single arg to the constructor, which is one step from being the type you already had

patrick.farwick05:07:38

Yeah, that is a good idea. I was looking at the docs for BufferedInputStream but was only seeing a read. Shouldve thought about how to convert it.

noisesmith05:07:11

yeah - start with the docs of the most specific thing first - the less specific thing won't even know the more specific thing exists :D

noisesmith05:07:25

that's a very reliable pattern in java

noisesmith05:07:50

that plus adding what should be new functions by defining a class extending the original (which is alien to clojure but predictable)

patrick.farwick05:07:52

Lol. Yeah, my experience is a bit weird because I have been learning a lot of Clojure lately but have not really done much java at all.

noisesmith05:07:38

the only time I've done java outside of clojure interop / extension was deliberately just in order to get better at clojure

noisesmith05:07:56

there's so many dumb things about java you never need to learn if you just use it from clojure

vale06:07:44

is there some simple way to convert a nested map like {"foo" {"bar" {"baz" 1}}} into a flat map like {"foo.bar.baz" 1} ?

markus.kiili07:07:05

(defn format [m] (let [[k v] (first m)] (if (map? v) (let [[rk rv] (first (format v))] {(str k "." rk) rv}) m ) ))

markus.kiili07:07:04

Seems to be working but maybe there is a simpler way.

noisesmith14:07:22

(defn compound-keys
  ([m] (compound-keys false m))
  ([prefix m]
   (into {}
         (map (fn [[k v]]
                (let [sk (if-not prefix
                           k
                           (str prefix "." k))]
                  (if (map? v)
                    (compound-keys sk v)
                    [sk v]))))
         m)))
#'user/compound-keys
(cmd)user=> (compound-keys {"foo" {"bar" {"baz" 1}}})
{"foo.bar.baz" 1}
(ins)user=> (compound-keys {"foo" {"bar" {"baz" 1 "quux" 2} "whatever" 9}})
{"foo.bar.baz" 1, "foo.bar.quux" 2, "foo.whatever" 9}

noisesmith14:07:24

note that that version breaks if a map contains more than one key

vale13:07:01

thanks for the suggestions! so the answer seems to be no... i have code about as complex as what you suggested. i was hoping there is a way to make it one line or something...

bitsapien16:07:48

(println (str "Hello Clojurians!" " Feels great to know a language that makes you think more, and code less. rich "))

mungaikamau717:07:51

Hi, hope everyone is well. I am trying to use partition-all as a transducer with a step but I some how get a ClassCastException.

ghadi17:07:21

user=> (doc partition-all)
-------------------------
clojure.core/partition-all
([n] [n coll] [n step coll])
  Returns a lazy sequence of lists like partition, but may include
  partitions with fewer than n items at the end.  Returns a stateful
  transducer when no collection is provided.

ghadi17:07:35

you're providing two args (n & step)

ghadi17:07:44

but the transducer flavor only accepts n

dpsutton17:07:27

i wanted that the other day 🙂 buts its not there

mungaikamau717:07:57

Is it possible to use partition-by to do the same thing?

mungaikamau717:07:20

Wait but the docs say that if you don't provide a collection it should return a transducer. Right?

dpsutton17:07:43

its the [n] arity. not for any particular arity

mungaikamau717:07:44

I assumed that is how the transducers work.

dpsutton17:07:19

ie, [n step] where you leave off the coll can't know that its not [n coll] which is just the normal two arity call

noisesmith17:07:45

I think for the three arg partition-all, a transducer can't really do much better than directly calling that on your input

noisesmith17:07:35

so you can spend less time on the problem and do that, or make the weird transducer if you can prove it's a bottleneck (unlikely imho)

nfedyashev18:07:54

Is there any simple way to stub session to return particular identity/user(eg :id 111 :name "Foo" :email "") in local dev environment? Basically, just trying to get rid of annoying manual logins after server/mount restart. An example of ring action/view/request handler:

(defn req-user [req]
  (get-in req [:session :identity]))

(defn foo-action [request]
  (if (req-user request)
    (do
      (perform-some-logic {:user-id (-> request :session :identity :id)})
      (layout/render request "authenticated.html"))
    (layout/render request "public.html")))
;; session-backend is buddy.auth.backends.session
(defn wrap-restricted [handler]
  (restrict handler {:handler authenticated?
                     :on-error on-error}))

(defn wrap-auth [handler]
  (let [backend (session-backend)]
    (-> handler
        (wrap-authentication backend)
        (wrap-authorization backend))))

(defn wrap-base [handler]
  (-> ((:middleware defaults) handler)
      wrap-auth
      wrap-flash
      (wrap-session {:cookie-attrs {:http-only true}})
      (wrap-defaults
        (-> site-defaults
            (assoc-in [:security :anti-forgery] false)
            (dissoc :session)))
      wrap-internal-error))
Still trying to make sense of the Ring middleware ecosystem. How can I approach this task? Should this be a middleware? Can I find any examples of how this could be done?

ghadi19:07:59

you need to parameterize your stack of middleware -- if you're in prod, do real auth, if dev, stub auth

nfedyashev19:07:05

hey @ thanks for your message. I wanted to start simple for now, just some local changes. But this feature would be nice to add eventually.

nfedyashev19:07:49

@hiredman thanks for the tip. I felt like Ring wiki docs are rather confusing for a beginner in Ring middleware like me. Mostly because of "out of context" examples like:

(use 'ring.middleware.session
     'ring.util.response)

(defn handler [{session :session}]
  (response (str "Hello " (:username session))))

(def app
  (wrap-session handler))
Just reading the source code of common ring middleware's were helpful at the moment. What I ended up with is this simple middleware:
(defn wrap-stub-session [handler]
 (fn
    ([request]
     (handler (assoc request :session {:identity {:email "" :id 777 :name "Baz Name"}})))))