Fork me on GitHub
#beginners
<
2017-11-28
>
hawari03:11:34

Hi, I would like to ask a question about how to layout your clojure code. Is it considered anti-pattern if I organize clojure code in this manner?

-- project
    -- service
        -- concern1.clj
        -- concern2.clj
    -- repository
        -- concern1.clj
        -- concern2.clj
Basically a service is the business logic part, where a data is transformed, validated, etc. Whereas a repository only concern itself with how to persist and retrieve a data. But I've found that it will most likely complect the business logic with the data storage, making the function not pure. Something like:
(defn store-something
  [something]
  (do-validation)
  (repo/store-something))
Can anybody give me an insight on this?

donaldball03:11:44

I’ve seen that pattern sometimes. It’s not imho a bad place to start if you’re new, though it may behoove you to look at your repository fns as they grow and see if there are useful commonalities that suggest something higher order.

hawari03:11:00

Do you mean that I just have to make sure that there's no logic/transformation happening in the repository layer @donaldball? If so, yes I'll make sure of that. The thing that made me ask this question is how am I supposed to test the service layer if there's another function involved? In the past I tend to do some "workaround" by setting the function to have multiple arity, basically "injecting" the repository function:

(defn some-fn
  ([data fn-to-store]
    (do-something-with-the-injected-fn))
  ([data]
    (some-fn data our-default-fn)))
Do you think such practice should be avoided or not?

hawari03:11:37

Apologies if my question seems trivial, functional programming, let alone using Lisp, is a fairly new concept to me.

donaldball03:11:11

I really meant: do the fns in repository/concern1.clj and repository/concern2.clj share sufficient commonalities that you could envision more general fns parameterized perhaps with rich data structures describing the repositories for concerns 1 and 2

donaldball03:11:33

And yes, I tend to do something very much like what you describe to inject pure fns instead of impure fns for testing. I tend to use protocols to encapsulate all my impure stuff so I would inject an impl of the protocol, but it’s morally equivalent.

hawari03:11:36

Ah, I get your point, yes the repository part tend to get similar, especially if it's just a simple CRUD application. Thanks @donaldball, I really appreciate your inputs.

max08:11:29

Can anyone help me figure out what I'm doing wrong?

max08:11:16

If there's just too much wrong in there, tell me, and I'll just follow a walk-through lol

max08:11:12

The error I'm getting is UnsupportedOperationException count not supported on this type: Long clojure.lang.RT.countFrom (RT.java:646)

phronmophobic08:11:14

it seems like your recur call has the argument order mixed up

phronmophobic08:11:37

looks like the 1st and 3rd args are swapped

max09:11:15

@smith.adriane Hmmm, I can see why the order would matter between (conj newcoll (first coll)) and (pop coll), but I don't understand why the incrementation of the counter has to be last in the recur call

phronmophobic09:11:11

the arguments to recur have to be the same order as the bindings for loop

phronmophobic09:11:29

so in this case, they have to be coll, newcoll, then counter

max09:11:40

revelation

max09:11:54

thank you, dear sir

phronmophobic09:11:39

also, the typical way to iterate through a collection with loop is something like

(loop [[n & numbers] [1 2 3 4 5],
       result []]

  (let [result (conj result (* n n))]
    (if (empty? numbers)
      result   ; we're done
      (recur numbers result))))

max09:11:04

I just reread the docs for recur. The rebindings happen in parallel at the end of the evaluation, so actually the conj and pop can be in any order too

phronmophobic09:11:47

so instead keeping a counter, you can use the destructuring

phronmophobic09:11:02

and checking with empty? to see if you’re done

max09:11:45

OK, thanks, I'll chew on that for a bit. I did refactor it without a counter, instead just using (= (count coll) 0) to see when I'm done, but now I have other problems. lol

phronmophobic09:11:58

the arguments to recur still have to match the binding order

max09:11:30

Oooh, empty? is better than (= (count coll) 0) ?

max09:11:16

right right, they have to match the binding order, but the order of evaluation in the recur won't affect the results, because the rebinding happens in parallel at the end

phronmophobic09:11:22

yes, empty? should be preferred over (= (count coll) 0)

rauh09:11:28

@smith.adriane That example would probably fail for the empty list

phronmophobic09:11:38

whoops, I copy and pasted that example from the docs to try to avoid dumb mistakes

phronmophobic09:11:40

but you’re right

phronmophobic09:11:45

it does fail for the empty list

phronmophobic09:11:32

(loop [[n & numbers] [1 2 3 4 5],
       result []]
  (if (empty? numbers)
    result   ; we're done
    (recur numbers (conj result (* n n)))))
would be better

rauh09:11:30

Now you're missing to add the last one

rauh09:11:53

@max @smith.adriane The general rule is: Don't destructure in loop. It'll almost always miss an edge case

admay15:11:19

I’ve never heard the general rule of don’t destructure in a loop but if you give me a compelling argument in favor of it I can add it to the guide 🙂 @ruah

rauh16:11:01

It's a little tricky: Destructing will call seq for you, but you need to still handle one case: What if you're passed the empty sequence? You can't test on the "rest" param of the destructing, since you might have a single value still to handle. And you can't test on the "first" param of destructing since values in the sequence can be nil. So you have to seq it initially and test the return values of that. That's what all loop/recur usages do in core. After the first loop you don't need to seq on the given sequence since the 'rest' will be nil'ed since destructing calls next.

admay18:11:44

If you’re passed an empty sequence in a destructuring binding say, (let [[x & xs] '()] ... ) then both x and xs will be evaluated as nil.

user=> (let [[x & xs] '()] (println "x: " x "\nxs: " xs))
x:  nil
xs:  nil
I’m not sure why this would be insufficient for testing the ‘truthiness’ of these values within a loop.

admay18:11:16

This seems to be fine for me.

user=> (loop [[x & xs] '() result []] (if (nil? x) result (recur xs (conj result x))))
[]
user=> (loop [[x & xs] '(1) result []] (if (nil? x) result (recur xs (conj result x))))
[1]
user=> (loop [[x & xs] '(1 2) result []] (if (nil? x) result (recur xs (conj result x))))
[1 2]

rauh18:11:32

@U405WJUPL It fails for the sequence that contains nil: [nil]

admay18:11:07

user=> (loop [[x & xs] '(nil) result []] (if (nil? x) result (recur xs (conj result x))))
[]

admay18:11:10

No it doesn’t

admay18:11:18

Regardless, that still wouldn’t be the fault of using a destructuring form within a loop. That’s a problem that ought to be address in the collection of data directly.

rauh18:11:20

yes, because why are you filtering out nil values? Maybe that's what you want to do, but you'll also filter out false values.

admay18:11:42

So then you change the check conditional to be something else.

admay18:11:57

Again, not a problem with using destructuring.

rauh18:11:03

In general use functions you probably want to process nil and false values. Unless you specify that in your function doc.

admay18:11:24

Still not a problem with the use of destructuring within a loop. This all ought to be handled by the if (... x) conditional or in the collection of data within the let binding, i.e. (let [[x & xs] (process-data '( ... ))] ... )

rauh18:11:39

You absolute need to test on your original value on the first iteration (seq xs) and then you can know if you've got an empty collection or not. It's impossible to do on [ x & r]

admay18:11:16

What do you mean by ‘it’s impossible to do on [ x & r ] ’?

rauh18:11:07

See my examples above, code that iterates over a collection should handle: [], nil, [nil] and everything else.

rauh18:11:14

For [] you absolutely NEED seq. For [nil] you can't check it anymore after destructing. You need to check (if (seq xs)) first.

rauh18:11:35

That's why all of clojure.core and cljs.core first does (loop [xs (seq xs)] ...)

admay18:11:11

‘For [] you absolutely NEED seq.’ We’ve already shown this to be false unless you’re talking about some specific edge case. Put together an example of what you are talking about (collection, wanted data, operations, etc…) and show me the failure of destructuring within the loop because I don’t see how any of the examples you’ve given so far are failed by destructuring.

rauh18:11:36

Again, you can do it with (loop [[x & r :as xs] (seq xs)] (if xs .... )) but it's ugly and confusing and calles seq twice.

admay18:11:02

Why are you calling (seq xs) to begin with?

admay18:11:26

Why not just (if (nil? x) ...) like I was doing above?

rauh18:11:27

because (if xs ...) is truthy when xs is []

admay18:11:51

So then call (if (nil? x) ... which would return false if xs is empty

rauh18:11:56

(seq []) will result in nil

admay18:11:14

And (nil? x) in that case will result in false.

rauh18:11:21

But (nil? x) is also true for [nil] so you can't do that

admay18:11:59

What is those are valid values?

admay18:11:16

If they aren’t, just filter them from the collection in the let binding

admay18:11:33

Either way, seq is not needed and destructuring in loops is not a bad practice

rauh18:11:00

If your goal is to iterate over every value in a collection then you have to it the way I explained. No other way around.

admay18:11:18

That’s just not true at all lol

rauh18:11:42

Well we'll just have to agree to disagree then 🙂

admay19:11:43

Give me an example of code that requires that and I’ll be more than happy to show you that it’s just 100% possible Also keep in mind that the clojure.core sticks to collections operations seq, vec, etc… by convention, not necessity nor optimization (per say).

rauh19:11:40

@U405WJUPL Pretty much every general use function requires it. I challenge you to rewrite select-keys with your method

admay19:11:31

That has nothing to do with destructuring. The original claim was that destructuring ought not be used within loop bindings.

phronmophobic09:11:09

(map #(* % %) [1 2 3 4 5])

rauh09:11:29

1. First (seq xs) the given argument and then check for nil 2. Then call next/first and use these.

rauh09:11:50

You can do it with destructing but you'll still have to call seq yourself and keep around the original one:

ajs15:11:35

I'm not sure I follow this, since you do not need to call seq when destructuring a vector. https://clojure.org/guides/destructuring

rauh16:11:37

It's a little tricky: Destructing will call seq for you, correct, but you need to still handle one case: What if you're passed the empty sequence? You can't test on the "rest" param of the destructing, since you might have a single value still to handle. And you can't test on the "first" param of destructing since values in the sequence can be nil. So you have to seq it initially and test the return values of that. That's what all loop/recur usages do in core. After the first loop you don't need to seq on the given sequence since the 'rest' will be nil'ed since destructing calls next.

ajs17:11:30

Why would you have a single value to handle if the sequence was empty?

rauh17:11:26

@U5YHNV0EA Sorry I mean the case when you have one item in the sequence.

ajs17:11:12

Are you saying that it in your first example, initial value of xs will be different depending on if you wrap the vector in an explicit seq?

ajs17:11:44

I would expect the :as test to be enough, without also calling seq

rauh17:11:43

You'll need the seq call in case you get passed an empty sequence, the seq will nil it. If you don't do that you'll have a truthy value and think you have one element.

rauh17:11:36

Check out the (macroexpand (let [[x & r] xs])) output. If you test on xs you'll have a truthy value unless you seq it explicitly. Like I said, I wouldn't use destructing in loop. Better to 1. Call seq on the sequence, 2. Test on that value. 3. Get first+next and potentially recur. That's also how all clojure.core does it.

eggsyntax20:11:08

@U5YHNV0EA what you might be missing there is that (seq my-sequential-thing) is the standard clj idiom for checking to see if your vector (or whatever other seqable thing) is non-empty. @rauh is suggesting that you use that standard idiom. (What @rauh was saying was totally correct; I'm just rephrasing in case it didn't come through clearly)

ajs21:11:06

I get it, I just found it an odd placement for 'seq' when combined here with destructuring. I typically just call seq on the :as arg directly, but that isn't as efficient either.

rauh09:11:04

(loop [[x & r :as xs] (seq [])
       result []]
  (if xs
    (recur r (conj result (* x x)))
    result))

rauh09:11:21

vs:

(loop [xs (seq [1 2])
       result []]
  (if xs
    (let [x (first xs)]
      (recur (next xs) (conj result (* x x))))
    result))

rauh09:11:57

The compiler will generate two consecutive seq calls in the first example. So slightly less efficient

phronmophobic09:11:41

seems like these would be great examples for the docs

rauh09:11:26

@smith.adriane I agree, it's tricky to write a correct loop. Especially if you also consider that sequences can contain nil values. There is lots of code out there that does it wrong. Feel free to add these explanations to the docs

rauh09:11:44

That's why in general any reduce (etc) is much easier to get right

phronmophobic09:11:24

I usually use reduce as well, with the main exception being doing stuff with core async

phronmophobic09:11:09

or loops that have side-effecty steps that would be awkward to write with doseq

max09:11:47

@rauh OK thanks 🙂

max09:11:02

this is great help guys

max09:11:36

I'm doing the koans starting with zero knowledge, and just reading docs. So I sometimes stray far from accepted patterns when the koans ask me to write big blocks of code

solf12:11:10

Parens of the dead (clojure video tutorials on web dev) were made 2 years ago, does anyone know if they are still relevant, or has clojure and tooling changed considerably meanwhile?

ajs15:11:35

I'm not sure I follow this, since you do not need to call seq when destructuring a vector. https://clojure.org/guides/destructuring

pawel.kapala16:11:05

Hello, is there a good way to construct map from one-to-many relation sql result? Or is there something ready that would do the trick? I’d like to consume sql query like this: SELECT * FROM foo f JOIN bar b ON (b.id = f.barid) into a map of a following form:

[{:id 1
 :some-foo-prop "foo prop value 1"
 :bars [{:id 1
            :bar-prop "bar prop value 1"}
           {:id 2
            :bar-prop "bar prop value 1"}
            ...]}
 {:id 2
  :some-foo-prop "foo prop value 2"
  :bars [{...}]}
 ...
]
Thank you in advance (looking if kormasql will be helpful right now)

Adam Faraj17:11:13

Can I upload a Clojure app using AWS?

val_waeselynck17:11:15

What does it mean to upload an app?

Adam Faraj17:11:47

Upload a web app

Adam Faraj17:11:52

like i can with React

val_waeselynck17:11:18

Yes sure, why would it be different? Either upload the compiled javascript, or the source and build in the cloud.

Adam Faraj18:11:32

I have no JS code. It's all in Clojure. But you're saying to use the Cloudfront services?

byron-woodfork18:11:49

Do you mean upload it via Elastic Beanstalk? ..If so, you can't upload it directly. You have to have it in a container.

Adam Faraj18:11:21

would lambda work?

byron-woodfork18:11:58

I believe there's another way to handle it as well via JAR file (not too familiar with this method). I'd give this a read first -- https://purelyfunctional.tv/article/jvm-deployment-options/

gonewest81818:11:17

I can’t tell from the question if we’re talking about a static website with some clojurescript, or if we’re talking about some backend service(s), or both…

gonewest81818:11:53

but if it’s the frontend piece, I deployed a test site built in reagent + semantic ui (soda-ash) literally 30 minutes ago. You compile the frontend piece to javascript and upload the resulting js, index.html, css, and any other static resources to a s3 bucket configured to serve a static website. http://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html

gonewest81818:11:04

on the other hand, if you’re asking about the backend piece, then your options include EC2, ECS or Lambda…. all depends on your requirements.

val_waeselynck20:11:52

As for the backend, I do use Elastic Beanstalk, using a Java setup + an uberjar

gonewest81821:11:30

Right now I’m experimenting with lein-clj-lambda to deploy in AWS lambda and proxy through the api gateway. So the frontend I mentioned earlier is a small test ui to drive the service.

fingertoe21:11:20

If I have a java example ` ProgramParameter[] parmlist = new ProgramParameter[5]; parmlist[0] = new ProgramParameter( 64 ); parmlist[2] = new ProgramParameter(8); etc... ` How would I build such an item with clojure interop?

noisesmith21:11:11

@fingertoe (into-array ProgramParameter (map #(ProgramParameter. %) [64 8 etc.]))

noisesmith21:11:35

alternatively (into-array [(ProgramParameter. 64) (ProgramParameter. 8) etc.])

fingertoe21:11:12

Thanks @noisesmith I will give that a go..

noisesmith21:11:21

the documentation for into-array plus https://clojure.org/reference/java_interop should also help - but I think either of those will give you the value you need in this specific case

phronmophobic22:11:49

oh, just noticed you edited the other example that I referenced earlier

phronmophobic22:11:04

maybe it’s worth consolidating that example? I think your tips were helpful >>> 1. First (seq xs) the given argument and then check for nil 2. Then call next/first and use these.

phronmophobic22:11:33

I think including some context for why destructuring in a loop is tricky since it seems like a common mistake

rauh22:11:00

@smith.adriane Nice. Personally, I really dislike the destructing example. It's confusing.

rauh22:11:15

But just an opinion on style...

phronmophobic22:11:30

yea, you’ve definitely convinced me to avoid the destructing in loops in the future

phronmophobic22:11:17

which is why I think including at least some background as to why it’s tricky is useful in the docs

phronmophobic22:11:28

since using the destructing version is tempting

drewverlee23:11:28

After putting a couple small clojure side projects under my belt, i'm starting to (this is going to sound crazy) question the value of naming things. I'm curious if it might not be more straight forward to almost forgo thinking about names. Spec the inputs to functions and just make a tool that can produce example inputs and outputs. Clearly this wouldn't all the time, but it might highlight where a name is better or worse then a couple examples. * this doesnt have much to do with clojure, i just tend to be more whimsical when writing in clojure.

phronmophobic23:11:49

sounds interesting, but I’m not sure what it would look like in practice

phronmophobic23:11:44

it seems like many arg names are actually fairly close to what you’re describing

(defn remove
  "Returns a lazy sequence of the items in coll for which
  (pred item) returns false. pred must be free of side-effects.
  Returns a transducer when no collection is provided."
  {:added "1.0"
   :static true}
  ([pred] (filter (complement pred)))
  ([pred coll]
     (filter (complement pred) coll)))

phronmophobic23:11:13

since pred and coll are fairly generic and not that different than a spec definition

drewverlee23:11:55

I was reflecting that my last project when slower then i wanted it to, i noticed i spent a lot of time moving code around in namespaces. I decided in the future i wasn't going to mess with multiple namespaces tell I had a more compelling story for how it was helping communicate intent. I sort of jumped from there to thinking about names in general.

sundarj09:11:15

i think you have run into the very common 'premature abstraction' problem, which is not specific to Clojure

sundarj09:11:07

naming isn't the issue here... it's abstracting code too early

drewverlee23:11:45

Like i have a namespace that gives meaning to a datastructure something like "animal" with functions like "pet". Of course, then half way through "animal" stops descibing the types of functions im putting in it. So i make another namespace, and now i'm juggling namespaces and thinking about what functions belong in what namespace.

drewverlee23:11:38

I think if i was focused 8 hours a day this might not seem like a big deal, but if you only have 30 minutes here and there to work on something, this seems to put stress on productively.

noisesmith23:11:41

@smith.adriane @rauh using loop to consume a collection in order is an anti-pattern in clojure, it’s very rarely the right thing

noisesmith23:11:45

and usually wrong

phronmophobic23:11:03

is there a way to we can improve the loop documentation?

phronmophobic23:11:46

it seems like it’s not an uncommon mistake to use loop incorrectly/poorly/non idiomatically

noisesmith23:11:04

hmm… yeah I can see the pedagogical value of showing how to consume a collection with loop though

noisesmith23:11:16

I would just recommend not using loop to consume collections

phronmophobic23:11:21

it’s used a lot in clojure core (mainly to create high order functions so you don’t have to write your own loop/recur)

phronmophobic23:11:44

like, is that reasonable usage?

noisesmith23:11:45

it would perform better as a reduce, but it would also mean putting multiple values into a hash-map accumulator and picking out the right value when it returns

phronmophobic23:11:29

it seems like loop is kinda in the same boat as macros. on the one hand, they’re usually not the right answer, but on the other hand, it can really make life easier when used correctly

noisesmith23:11:13

yeah, my main point is that if you consume a collection in front to back order (and especially if you always consume the entire collection) a reduce does the same thing but performs better and avoids tricky boilerplate and bugs with (if remains …) as opposed to (if (seq remains) …)

phronmophobic23:11:17

regarding the reagent example, does packing and unpacking an accumulator still perform better than using loop/recur?

noisesmith23:11:39

it would be worth benchmarking

phronmophobic23:11:49

i agree with you about using loop just to consume a collection, but I usually prefer using loop in an example like reagent

noisesmith23:11:02

and it would be a question of calling assoc at each step (or even assoc! on a transient)

noisesmith23:11:22

IMHO multi value accumulators are a normal way to use reduce

noisesmith23:11:42

I might just make a reduceN that allows N accumulator bindings… that would be fun actually

noisesmith23:11:57

(and I’d make it avoid the expense of packing/unpacking a hash map, of course)

phronmophobic23:11:58

I think providing rules of thumb and idiomatic usage is useful to include in the documentation, especially for something as fundamental as loop

noisesmith23:11:09

that makes sense, yeah

noisesmith23:11:39

I might take some time eventually to at least note on clojuredocs to consider using reduce (potentially with a complex accumulator) if consuming a collection in order

noisesmith23:11:10

ooh! reduceN could accept N accumulators and N input collections (like map does) - that would be cool

phronmophobic23:11:59

I’ve learned a lot just from the discussion about loop so far

phronmophobic23:11:42

do you still use reduce over loop if you’re performing side-effecty stuff in the body?

noisesmith23:11:09

only if I’m still using the accumulator

phronmophobic23:11:11

obviously, if there’s no accumulated state, doseq would be better

noisesmith23:11:18

if I don’t even need an accumulator, I use doseq

phronmophobic23:11:54

ah ok, I think i’ve used loop because it works with core/async