Fork me on GitHub
#beginners
<
2020-02-21
>
Michael Stokley00:02:06

what's an idiomatic way to thread a nilable value through multiple functions? should i use multi-arities thoughout or?... (def f1 ([] (f1 nil)) ([x] (if (nil? x) (f2) (f2 x)))) (def f2 ([] (f2 nil)) ([x] (...)))

noisesmith00:02:01

weird idea: (apply f2 (remove nil? [x]))

noisesmith00:02:10

probably not what you want haha

Michael Stokley00:02:14

but i'm worried about being legible rn

hindol03:02:50

There is some-> which continues threading if the result of previous call is non nil.

Michael Stokley00:02:02

alternatively, i guess i could pass it through and only interrogate it at the terminal

Michael Stokley00:02:23

like this

(def f1 [x] (f2 x))
(def f2 [x]
  (if (nil? x) ... ...))

didibus00:02:03

A lot of functions will naturally compose to return nil when given nil

didibus00:02:40

But if yours doesn't, ya, whatever way you want.

didibus00:02:23

But if you're concerned about threading, and not really about designing a fn that returns nil on nil. You could look into using some-> and some->> instead

Michael Stokley00:02:33

i've never heard of those, thank you for the heads up

Mario C.00:02:38

Im working on an application that has degraded to a mess. It uses Apache POI as one of its main dependencies. (Basically acts as a wrapper for it) The users uploads an excel workbook and then use the web app to input values, the application evaluates the spreadsheet and returns the outputs. I am using VisualVM to try and pinpoint where exactly the bottleneck is and to my surprise none of the functions that I suspected are showing up. This could be because I dont know how to use VisualVM. Here is a screenshot of the result

noisesmith00:02:34

the two clojure.lang.reflector methods nearest the top should be combined - because the same fix eliminates both, and when combined they are near the top (under what is essentially polling that shows up looking like work)

noisesmith00:02:47

waiting on a promise-deref / poll / select are all basiically polling, so shouldn't be optimized

noisesmith00:02:18

*warn-on-reflection* will usually help you eliminate the reflection overhead

user=> (doc *warn-on-reflection*)
-------------------------
clojure.core/*warn-on-reflection*
  When set to true, the compiler will emit warnings when reflection is
  needed to resolve Java method calls or field accesses.

  Defaults to false.
nil

Mario C.00:02:00

I got a wall of orange text! Let me fix those reflection warnings. Good catch, perhaps this is the golden ticket

seancorfield00:02:52

I've gotten into the habit of putting (set! *warn-on-reflection* true) immediately after my ns form in almost every new file these days 🙂

seancorfield00:02:15

Maybe I should put them in the clj-new templates? 🙂

noisesmith00:02:58

I learned the other day that there's a clj / deps.edn version of lein check, I wonder if that turns on reflection warnings

noisesmith01:02:02

@mario.cordova.862 after coming back and looking again, it looks like a significant amount of time is spent on file system ops - I don't know your domain, but maybe active objects don't need to be files on disk? A common mistake is thinking that since the thing you work on is commonly refered to as a "file", that it needs to exist on a file system while your app works on it

noisesmith01:02:35

but then again it could be that POI is inflexible and can't use in-memory data (?)

noisesmith01:02:49

eg. once I worked on a webservice that resized files for dynamic frontends, and we eliminated most of our turnover latency by doing all the image ops in memory and not using the disk except for caching

Mario C.01:02:06

That could be the case but I am not familiar enough with the codebase to know enough about what exactly is going on. As far as I am aware the WorkBook object should be in memory.

Mario C.01:02:01

I know we are doing something wrong because our operations are taking 2-3 mins

tjb04:02:52

hey team! is there any preferred directory structure for clojure?

tjb04:02:49

example:

src
|-> user
    |-> repository.clj
    |-> controller.clj 

seancorfield04:02:55

@tjb Do you mean beyond the resources/`src`/`test` top-level part?

tjb04:02:59

correct!

seancorfield04:02:20

Not really. Different people -- and different companies -- prefer different things.

tjb04:02:27

ok cool!

tjb04:02:42

ima role with the one i posted above

seancorfield04:02:01

I tend to separate things by "domain" concepts more than "function", although within a pure sort of MVC web app, the core is often broken into handlers, model, views or some such.

tjb04:02:22

where would the controller live in that case?

tjb04:02:29

under model?

seancorfield04:02:27

controller = handler

seancorfield04:02:56

I'd use controllers in other languages but Ring talks about handlers so...

👍 4
tjb04:02:57

and model would be DB calls then i assume

tjb04:02:03

is there such thing as a service layer?

seancorfield04:02:58

I've never been a fan of a generic "service" layer. I think that is a bit artificial.

tjb04:02:51

would you throw those types of methods in the handler?

seancorfield04:02:11

I think you'll see some sort of "store" section in a lot of Clojure apps but I think, overall, things are a lot more organic.

seancorfield04:02:15

I tend to start with one function per "request" (or "route") and group "related" functions together in a single handler... so maybe ws.admin.handlers.photos would be all the photo-handling functions.

seancorfield04:02:05

And each of those would start off doing all of whatever is needed and then I organically refactor into smaller functions as needed and shuffle those off into different parts of the domain and/or "store" or whatever.

tjb04:02:25

thats a v cool method

seancorfield04:02:30

At work, we tend to evolve our structures quite a bit over time.

seancorfield04:02:14

We have a monorepo now, with about thirty subprojects. Some of those are sort of "core" function for different parts of our overall system, some of them are actual "web apps".

seancorfield04:02:37

Some of our apps don't have the handlers/model/views separation, some do. Kind of depends on what the app looks like -- whether it has a small number of generic views or a larger number of specific, more complex views. Same with the handlers and "model" aspects.

seancorfield04:02:05

Here's a concrete example of one app that doesn't have that breakdown:

ws
└── seo
    ├── geo
    │   ├── affiliate.clj
    │   ├── core.clj
    │   ├── data.clj
    │   ├── listing.clj
    │   ├── locale.clj
    │   ├── page.clj
    │   ├── photo.clj
    │   ├── profile.clj
    │   └── search.clj
    ├── geo.clj
    └── sitemap.clj

👍 4
tjb04:02:12

very interesting! i like that there is no strict way that a project must be structured

seancorfield04:02:59

And then here's another app that is much more traditional MVC-structured:

$ ls ws/admin*
ws/admin.clj

ws/admin:
handlers  model  screening.clj  views  web

seancorfield04:02:23

Main entry points are ws.seo.geo above and then ws.admin below.

tjb04:02:48

actually i like that structure

tjb04:02:52

i may mimic it!

seancorfield04:02:11

The admin app has 30 handler namespaces and 16 model namespaces.

seancorfield04:02:24

The SEO Geo app is the total code shown.

tjb04:02:31

model though is not in the sense of a java model correct?

seancorfield04:02:06

"domain model"?

seancorfield04:02:41

It has namespaces corresponding to separate aspects of the domain model, yeah, but not "objects"/"classes".

seancorfield04:02:01

the handlers correspond instead to the sections of the web app itself, not the domain.

tjb04:02:48

thank you for another clojure lesson @seancorfield!

seancorfield04:02:27

Feel free to hit me up in DMs any time you have Qs.

seancorfield04:02:41

(I can share more of how we do stuff in private)

tjb04:02:53

thanks a ton sean!!

tjb04:02:57

i may in the future 🙂

johnj06:02:51

what's the resources dir typically used for?

seancorfield06:02:41

Non-code stuff -- CSS JS images

lepistane09:02:55

Hello i've gotten very used to luminus and how it operates BUT i have no idea how to setup 'batteries' and create my template. I started with lein new app X and i'd like to grow it to have 'batteries' similar to luminus (clj, cljs, db, api, reloaded workflow....) are there any resources/tutorials or something i could have a look at this that would get me started or give me hints how to start ? for instance i have trouble with reloaded workflow I have added ring, jetty i cider-jack-in and the app works - great but when i make changes i have to cider-quit then jack-in again in order to have changes how do i make it nice so that jetty uses handler that is changed once it's changed hopefully my questions make sense

Crispin09:02:58

you can re-eval the defed function inside emacs

Crispin09:02:05

or re-eval the entire namespace

Crispin09:02:39

c-c c-k i think??

lepistane09:02:08

yes but that doesn't do it and i don't wanna do that manually all the time. I'd like to save file and it's reloaded (just like in luminus)

Crispin09:02:12

luminus includes a reloading middle ware. You could set up a similar thing. I just c-c c-k. Its the same number of keypresses as c-x c-s 😉

Crispin09:02:46

note that server side luminus last time I used it didnt reload immediately, but on next HTTP request

Crispin09:02:12

which meant if you were working in an async system with websockets, saving the file didnt hotload the code

Crispin09:02:26

so I got used to telling cider to re-load the namespace

lepistane09:02:51

oki this makes sense

lepistane10:02:09

@U1QQJJK89 do you have any resources to recommend that i could use to figure out reloaded workflow or atleast what's out there? i saw some people are using namespace.tools

Crispin10:02:53

I would just check the luminus template, see what they are using to start out

seancorfield17:02:52

If you get into the habit of eval'ing every change you make (e.g., change a bit of code, eval surrounding/top-level form), then you don't even need to worry about saving files to get the REPL caught up to the latest state.

seancorfield17:02:45

That's the work folks like Eric Normand and Stu Halloway recommend for REPL-Driven Development -- and I've found it to be a really good practice.

seancorfield17:02:06

I've never liked any of the auto-reload/refresh workflows and with that RDD workflow, I just don't need them.

👍 4
jumar08:02:13

I tend to prefer what Sean already said. One situation when I've been using tools namespace refresh is when switching git branches (I did that lot in the past few days). Also, it seems that @U45SLGVHV may need to pass in the actual var ( #' ) instead of just plain symbol

mloughlin16:02:48

condp seems like a weird little function

Ankon Chakraborty06:02:36

That is just a keyword nothing special

Ramon Rios17:02:44

Hey everyone! Does exist a function for compare list values?? I tried to use compare , but it does not work with a list

Ramon Rios17:02:27

😅 i should had done this before

Ankon Chakraborty17:02:38

= is enough for any comparisons in clojure. 🙂

Ramon Rios17:02:42

Not the one that i would. But i can try convert my list to sets and then make it work.

Ramon Rios17:02:16

According to the documentation, two lists with same elements but not in same order are not equal when using = . The same does not apply to a set. So probably my approach will turn this list into a set to = compare them.

Ankon Chakraborty17:02:46

Well if you wanna check the equality you should be checking the order also. But yeah on the other hand I don’t know your use case. So if you wanna check if all the elements are there or not then yeah. converting into set works.

Ankon Chakraborty17:02:08

or if you don’t wanna convert your list to set you might wanna do something like this

(every? #(.contains [1 2 3] %) [2 1 3])

Ramon Rios17:02:46

My case is: i'm doing a etl, so now i'm checking if the ids on the old database are available in the new

andy.fingerhut17:02:54

If you want equality of the set of elements, ignoring order, then elements of a set and = are things Clojure provides to do that for you. There are other ways, too, of course, but if you prefer to use something built in, a set is a good way to go.

andy.fingerhut17:02:46

At least, if all of that fits in memory at one time 🙂 If not, then you either need to find a bigger memory machine, or do it in some other way.

Ramon Rios18:02:29

So i do a select in both bases, do a map for get the id of each element and then turn into a set

Ankon Chakraborty18:02:25

Doing the every? thing will help you certainly

Ramon Rios09:02:37

This solution will be better than transform to another data structure just for compare values

Ramon Rios09:02:46

I just would like to understand something with it: using every? , what i saying is: for every element in one list, apply this anonymous function that check if this element exist in the other list. Am i right?

Ankon Chakraborty07:02:33

Yeah it takes a predicate and checks if the predicate is true for every element in the list

andy.fingerhut07:02:20

If you want equality of the elements of two collections, viewed as sets, you would have to do the every? in both directions: is every element of A in B, and is every element of B in A? If you only do it in one direction, e.g. is every element of A in B, then you have shown that A is a subset of B, but that doesn't necessarily mean they are equal sets.

Ankon Chakraborty08:02:32

The thing is he doesn’t wanna do it on set he wanna do it on lists. And he wanted to check it in one direction only. That’s why I suggested changing the data structure just to compare is an overhead

tjb17:02:19

ooooo!!!

tjb17:02:23

thank you for the ping sean!

tjb17:02:45

ok cool i was going with this too by having directories by domain

tjb17:02:56

so my in case a directory would be called patent

tjb17:02:14

which, if i am understanding correctly, lines up with Rick’s comment

4
Michael Stokley18:02:29

is it typical to use a mocking library in clojure or do folks stick with with-redefs and write their own "fakes"?

Michael Stokley18:02:43

(i feel like i have to question every convention i'm familiar with from java land)

noisesmith18:02:50

there's a strong preference to stubs over mocks, but there's libs that do both

Michael Stokley18:02:50

thank you @noisesmith i'll go in that direction.

noisesmith18:02:00

also, the usage of immutable data and "mostly pure" functions means doing a lot less of either

noisesmith18:02:03

if done properly

noisesmith18:02:52

by "mostly pure" I mean segregating things that must do mutation (eg. IO) so that they touch as few other parts of the system as possible

Michael Stokley18:02:23

that's definitely my intention. fewer tests and segregating pure/impure functions

noisesmith18:02:56

I don't know about fewer tests, but definitely less need for mocking/stubbing when you use immutable data and non-side-effecting logic

noisesmith18:02:33

most tests should look like (is (= {some data literal here} (f some input literal)))

👍 4
Michael Stokley20:02:25

i'm having some trouble using with-redefs across multiple namespaces. in the source code, i import a library (`clj-http :as http`) and then invoke http/get . can with-redefs target that get function, or should i wrap it in the source code like (def get http/get)?

Michael Stokley20:02:19

it looks like there is a clj-fake-http lib for this, but i'm interested in working it out myself if possible

Michael Stokley20:02:19

maybe another option is to import the lib using refer like [clj-http :refer [get]) but then i'm shadowing get

seancorfield20:02:09

with-redefs can use qualified symbol names.

Michael Stokley20:02:44

does qualified mean namespace qualified? naively, i've tried (with-redefs [sut/http-client/get stub-http-get but that throws unable to resolve var

seancorfield20:02:16

(with-redefs [http/get stub-http-get] ,,,)

seancorfield20:02:26

Just like you'd use http/get in your own code.

Michael Stokley20:02:28

aha, ok. that is working. thank you!

Michael Stokley20:02:35

(for a minute it looked like the real call was still going through, but it was an un-stubbed underlying call)