Fork me on GitHub
#beginners
<
2019-08-07
>
GC00:08:45

Hi, I am not very clear, why liberator.core/defresoure is used for??

GC00:08:11

Plus, do I need to them too alongside functions?

GC00:08:09

my second question is, can we call functions inside liberator?

noisesmith00:08:37

defresource is for serving content out of a jar that started as a file in your repo before building

noisesmith00:08:15

so if you want a static file that isn't represented by code or a string in a source file, and want to be able to serve it directly from the jar without unzipping or whatever

noisesmith00:08:22

commonly used for index.html

noisesmith00:08:58

liberator is a collection of http libs, you typically call whatever functions you need in order to serve a request from inside your handler function for an individual route / method

noisesmith00:08:36

correcting the above: that liberator calls a "resource" isn't what we'd usually call a resource - it's a way ot describe an endpoint (and can include arbitrary function calls)

GC00:08:16

so, they are just defined for end points ?

noisesmith00:08:21

I'm sad they decided to use the word "resource" for this, as it already has a meaning in the clojure http handling world :/

noisesmith00:08:39

yeah, they define endpoints that can be called if a certian route is hit

noisesmith00:08:41

@gagan.chohan this intro shows using an arbitrary function for :handle-ok (the non-error case) http://clojure-liberator.github.io/liberator/tutorial/getting-started.html

noisesmith00:08:57

(defroutes app
  (ANY "/foo" [] (resource :available-media-types ["text/html"]
                           :handle-ok (fn [ctx]
                                        (format "<html>It's %d milliseconds since the beginning of the epoch."
                                                (System/currentTimeMillis))))))

noisesmith00:08:06

and of course that function can call other functions

GC00:08:01

cool, Thanks for the detailed explanation.

GC01:08:51

is there a way to check in testing that what functions did function being tested called?

noisesmith01:08:37

not that I know of (it may well exist in a third party lib I haven't used), clojure (and fp generally) tends to lead you toward using example based and property based tests that are focused on input values and return values, and not the control flow of the code executed

GC01:08:02

There’s a function that calls S3 or cache to retrieve data, I want to test both scenarios, in case of S3 and cache, they both call different functions

noisesmith01:08:04

come to think of it, I have seen this done with with-redefs

GC01:08:08

output will be same in both cases

GC01:08:53

so, what do you recommend to test such a function

noisesmith01:08:51

I take back what I said before: the pattern is (let [executed (atom [])] (with-redefs [some-ns/call-s3 (fn [& args] (swap! executed conj {:f :some-ns/call-s3 :args [args]}))] (some-other-fn))

noisesmith01:08:28

after you execute some-other-fn you check the contents of executed (it could contain zero or more entries of course)

noisesmith01:08:42

and you would "redef" every function you want to track / stub

noisesmith01:08:15

I'm personally not a fan of this sort of test, and with-redefs shouldn't be used in a multi-threaded context, but it does work

GC01:08:17

so , is there any preferred approach to test what is being called, S3 or Cache, considering both of them returns same data

noisesmith01:08:47

that's what the above will do (add another redef for other-ns/hit-cache)

noisesmith01:08:16

it replaces the function you want to track with something that records its invocation - which is then data you can make an assertion about

GC01:08:44

okay, I will first look at with-redefs docs

noisesmith01:08:51

@gagan.chohan

user=> (def calls (atom []))
#'user/calls
user=> (with-redefs [clojure.core/println (fn [& args] (swap! calls conj {:f 'println :args args}))] (println :OK))
[{:f println, :args (:OK)}]
user=> @calls
[{:f println, :args (:OK)}]

noisesmith01:08:47

it shouldn't be used with multiple threads in use, because two overlapping with-redefs calls can lead to losing the original function definition

noisesmith01:08:07

so you're committing to single-threaded execution during tests if using this

noisesmith01:08:30

(or very careful usage of threads and redefs)

dpsutton01:08:37

Difficulty in testing is (to me) a sign that you need to refactor. If you can change this to have two (or arbitrarily many) strategies to do whatever you might get more adaptable code and more easily tested code at the same time

Godwin Ko01:08:02

don’t know why this doesn’t work (map #(.autoSizeColumn sheet %) (range 2)) while call it one-by-one (.autoSizeColumn sheet 0) & (.autoSizeColumn sheet 1) works?

beders02:08:31

what error are you getting?

dpsutton02:08:47

try doall or mapv? perhaps laziness is involved?

👍 4
4
Godwin Ko02:08:17

you save my day, thx a lot!!

👍 4
dpsutton03:08:06

be wary of side effects from a map

Crispin04:08:25

does .autoSizeColumn return a new value? If its all just for side effects, use doseq instead.

Crispin04:08:53

or, as you are looping over indexes, have a look at dotimes

byrongibby06:08:54

Hi. After setting JAVA_CMD to C:\Program Files\Java\jdk1.8.0_211\bin\javac.exe running lein install produces

javac: invalid flag: -Dclojure.compile.path=C:\git\xx\mranderson/target/classes
Usage: javac <options> <source files>
use -help for a list of possible options
Does anyone know what I'm doing wrong/how to fix?

byrongibby06:08:27

JAVA_CMD should have been set to C:\Program Files\Java\jdk1.8.0_211\bin\java. 🙂

4
Godwin Ko08:08:37

I try to export a large excel file using docjure, and the jvm keep crashing due to OutOfMemoryError, GC overhead limit exceed. Which jvm options should I alter? Thx in advance 🙇:skin-tone-2:

schmee09:08:55

try -Xmx2G -Xms2G and replace 2G with however much memory you want!

4
Casey08:08:07

often i'm faced with making a serious of changes to a map (as part of some data munging), I generally use the -> thread macro to change update/assoc ops, however it gets awkward when I want to , for example, change/add a key based on the value of another key. is there no function in cloure core for doing this?

Casey08:08:14

i can workaround it by using as-> but .. 😕 i don't see that macro used much in code in the wild vs ->, which makes me think i'm missing something

Crispin10:08:29

I just use destructuring in let's in combination with the threaded assoc/updates or bind to one var (usually it's %) and use get-in's.

Crispin10:08:06

like

(->> collection
     (map #(let [{:keys [one two]} %]
             (-> %
                 (assoc-in [:key :path 1] (get-in % [:other :key :path]))
                 (assoc :two one)
                 (assoc :one two)
                 (update-in [:yet :another] inc)))))

Crispin10:08:42

you can also destructure in the (fn ...) arg list instead of using anonymous inline func

Crispin10:08:15

(->> collection
     (map (fn [{:keys [one two] :as arg}]
            (-> arg
                (assoc-in [:key :path 1] (get-in arg [:other :key :path]))
                (assoc :two one)
                (assoc :one two)
                (update-in [:yet :another] inc)))))

Casey10:08:25

thanks for sharing. using destructuring to bind the values before threading is nice

schmee09:08:20

@ramblurr Clojure core is intentionally limited when it comes to stuff like this, so I recommend either writing your own helper functions or to check out something like https://github.com/nathanmarz/specter/ and see if that might be of use

magthe10:08:20

I have a dev scenario / lein question. Say that I have two Clojure projects, libfoo and svcbar. svcbar has libfoo as one of its dependencies. Now I want to make a change in libfoo and try it out in svcbar. I bump the version i libfoo from 1.1 to 1.2-SNAPSHOT, I test, I build a JAR -- I'm happy. Then I change over to the svcbar dir and change its dependency on libfoo to 1.2-SNAPSHOT, but of course that build fails because that version isn't published. How do I make a locally built version of a library available to lein when building something that depends on it?

Crispin13:08:50

You can lein install it to install it to your local maven repo

Crispin13:08:03

or you can use "checkouts" so you don't even need to install it

Crispin13:08:33

oh. just saw this was answered below... heh.

magthe14:08:49

What is a Maven repo? Is the cache in ~/.m2 actually a full-blown Maven repo?

mafcocinco12:08:47

@magnus_clojurians there are 2 ways: if you are done with dev work on libfoo you can run lein install from the root directory of libfoo on your dev machine. This will install the new version of libfoo on your local machine and should allow svcbar to find it as a dependency. If you are still working on libfoo and want to dynamically pick up changes in svcbar leiningen has a feature called checkouts. You can set up a directory in svcbar that has a symbolic link to libfoos project root. Lein should use this as a source for dependency resolution. The main advantage to this approach is you don’t have to run lein install every time you make a change to libfoo. But both approaches should be the same otherwise.

magthe12:08:41

I've seen lein install, but it refers to lein deploy which only refers to publishing to public repositories. Is there some documentation on how to use it locally? I've never seen anything about checkins, it's not a command I guess... so where can I find information about how to actually use it?

mafcocinco12:08:17

Correct. It’s actually checkouts and it would be a sub directory of svcbar. You should be able to find the documentation here if you search the page: https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md. Sorry I couldn’t find a direct link, responding from my phone. 😳

4
mafcocinco12:08:31

Also, deploy does push the Jar but I think install simply installs it on the machine where the command is run. Could be wrong about that but I think that is how it works.

magthe12:08:01

Thanks, that ought to get me started 😃

👍 4
Noah Bogart14:08:49

is reduce lazy? It doesn't look like it, but my experiments are proving inconclusive

dpsutton14:08:21

what data points to it being possibly lazy?

Noah Bogart15:08:50

on second thought, i (naively) assume every iteration tool in clojure is lazy. the language does a good job with being consistent on that front (all of the seq-related tools are lazy), and the reduce docs don't say either way. the impl isn't wrapped in a lazy-seq, which points to it not being lazy

Noah Bogart14:08:37

my bad programming, i think

Noah Bogart14:08:21

i had a doseq, and wanted to save the results, so i switched to a for loop wrapped in a doall, but on closer inspection i was seeing some weird behavior, so i thought i'd try using reduce to make sure, and then weird behavior persisted, so the problem looks to be an error in my logic somewhere

Noah Bogart15:08:45

thanks for the quick answer!

gerred16:08:02

is there a reason at this time not to just openjdk 12?

gerred16:08:07

and prefer openjdk 11

seancorfield16:08:54

8, 11, 14 are the LTS (Long Term Support) versions which is why folks tend to prefer those versions, even in the openjdk camp.

gerred16:08:21

ahh 👍 yeah, ok. for some reason in my head I was thinking 12 was LTS

gerred16:08:48

time to add in the adoptopenjdk cask then. 😄

seancorfield16:08:23

We're stuck on openjdk8 because some pieces of our infrastructure won't run on jdk9+ but I'm testing a lot of stuff on both jdk8 and jdk11 at this point.

Alex Miller (Clojure team)17:08:46

for Clojure, we're trying to emphasize the LTS versions so we recommend 8 and 11 as primary right now (but the matrix tests with 12 too)

Alex Miller (Clojure team)17:08:28

we are making no effort to test with 9 or 10 right now (although we did prior to 11 and there are no particular issues with them)

Ashley Smith22:08:26

I need some help - I'm doing my first ever POST request and so I'm a little unsure what to expect. I have a successful connection between API and website but the data I POST just seemingly disappears. Its the end of my night and I feel like I don't know where to go from here. Here's a gist with my code and the output I've got from print statements: https://gist.github.com/Ashe/fa13a7a1ee44736676b7116663f8f751

pberganza22:08:37

I don't use Compojure but I think I see the issue: In the line 32 of core.clj you have

(c/POST "/forum/submit" post (db/submit-forum-post post))
But in this case, post is not the parameters you sent in the request. It is the whole request object. If you wanted the params you'd have to destructure it like:
(c/POST "/forum/submit" {params :params} (db/submit-forum-post params))

Ashley Smith22:08:51

If you look at the gist I sent, the params inside the request is empty - I was trying that at first but I thought for the sake of getting some help with this I'd show the whole request in the print statement

pberganza22:08:39

Another thing I noticed is that you're not ussing any middleware. If you expect JSON you should probably use something like https://github.com/ring-clojure/ring-json If you're using url encoded forms there is already a function on ring.middleware.params called wrap-params that handles that.

pberganza22:08:12

@ULLSSKY00 Yep! That's due to not using any middleware to handle the request.

Ashley Smith22:08:14

Yeah I don't really understand any of this middleware stuff - so is that why my data is just disappearing?

Ashley Smith22:08:21

okay getting somwhere

Ashley Smith22:08:41

Do you know how hard it'd be to get that middleware up and running? Do I wrap it like I do with cors or something?

pberganza22:08:17

It is not dissappearing! It is on the :body property IIRC. It's just not in a format you can use. You're already using a middleware! wrap-cors is exactly that.

pberganza22:08:39

So you could write (wrap-params (wrap-cors router .....)) Or in a more readable way:

(-> router
    wrap-params
    (wrap-cors :access-control.......))

Ashley Smith22:08:42

trying that ring/json middleware

👍 4
Ashley Smith22:08:08

I don't really like that syntax unless the brackets get reaally bad

pberganza22:08:26

It's much more readable when you get used to it 😅

Ashley Smith22:08:16

;; Wraps middleware around router
(def api-handler
  (rjson/wrap-json-response
    (cors/wrap-cors router
      :access-control-allow-origin [#""]
      :access-control-allow-methods [:get :put :post :delete])))
Does this look okay? As my data still doesn't appear in my print statemnt

Ashley Smith22:08:29

sorry if im missing something, I just really want to get this done before bed if its a simple thing

Ashley Smith22:08:26

Thank you ❤️

Ashley Smith22:08:33

Thank you thank you thank you! ❤️

pberganza23:08:07

Congrats ❤️ and good night for you haha 😅

Ashley Smith23:08:50

I got past that problem, my current problem is with postgresql

Ashley Smith23:08:19

2019-08-08 00:06:21.167:WARN:oejs.HttpChannel:qtp878530251-18: /forum/submit
org.postgresql.util.PSQLException: This ResultSet is closed.
	at org.postgresql.jdbc2.AbstractJdbc2ResultSet.checkClosed(AbstractJdbc2ResultSet.java:2654)
	at org.postgresql.jdbc2.AbstractJdbc2ResultSet.setFetchSize(AbstractJdbc2ResultSet.java:1771)
	at org.postgresql.jdbc4.Jdbc4Statement.createResultSet(Jdbc4Statement.java:39)

Ashley Smith23:08:03

at least the POST is out of the way!

pberganza23:08:13

That could mean you're closing the connection to postgres before you read from the ResultSet 😮

pberganza23:08:00

Do you have a gist for that?

Ashley Smith23:08:03

I'm afraid I'm not quite sure of what you mean, but if you mean what the actual statement is here you go https://gist.github.com/Ashe/ff245da7595a7afde100d896e3dc64ce

Ashley Smith23:08:06

thank you for trying to help!

Ashley Smith23:08:20

I cannot wait to sleep but if I don't finish this I will be awake all night on my laptop trying to do it

pberganza23:08:33

Don't worry, I have some free time at work today so this keeps me busy parrot give me a minute to check that

Ashley Smith23:08:12

its my first time working with all these things so hopefully its just an amateur mistake you're looking for 🙂

pberganza23:08:45

Is this where you are getting the error? 😮 I can't really find anything wrong with this.

Ashley Smith23:08:08

error triggers when I do submit-forum-post

Ashley Smith23:08:01

It says the error comes from opening and closing the connection or something but there's only two things I do with SQL - my GET for the forum posts and pages (done on another page) and this POST

pberganza23:08:17

Yeah. It works on my end 😮 (I made a dummy db).

Ashley Smith23:08:33

and the get stuff works fine even now. Commenting out the sql/insert! stuff makes this work

Ashley Smith23:08:40

let me try deleting my db maybe?

pberganza23:08:10

It all works for me here 😮 and I can't see anything obvious

Ashley Smith23:08:28

CREATE TABLE Tags (
  TagID bigserial NOT NULL,
  TagLabel varchar(18)
);

CREATE TABLE Users (
  UserID bigserial NOT NULL,
  Username varchar(18) NOT NULL,
  UserHandle varchar(12) NOT NULL UNIQUE,
  JoinDate timestamptz NOT NULL,
  IsAdmin bool NOT NULL,
  PRIMARY KEY (UserID)
);

CREATE TABLE Posts (
  PostID bigserial NOT NULL,
  PosterID bigserial,
  PostDate timestamptz NOT NULL,
  PostTitle varchar(100) NOT NULL,
  PostContent text NOT NULL,
  PRIMARY KEY (PostID),
  FOREIGN KEY (PosterID) REFERENCES Users(UserID)
);

Ashley Smith23:08:32

These are my tables

Ashley Smith23:08:45

is there something in the tablet that's making it crash?

Ashley Smith23:08:43

Oh- could it be that I'm not passing PostDate? I was expecting that to just insert the current time in though?

pberganza23:08:35

That does give me an error 😮

Ashley Smith23:08:10

If only I knew how to pass current time in

pberganza23:08:37

You can add DEFAULT NOW() to the table definition.

Ashley Smith23:08:52

Just tag it on after not null?

pberganza23:08:01

Yep, that's it 😉

Ashley Smith23:08:37

hmm the error is still there

pberganza23:08:42

Another thing, PosterID is defined as a bigserial. Which means it will try to autoincrement whenever you create a new record. I imagine you want to add it yourself, so bigint would be a better type

Ashley Smith23:08:13

Does it not have to match the same type as the other table? It's like linked to the Users table?

pberganza23:08:21

(But then you'd have to send PosterID from your API).

pberganza23:08:23

They are! int and serial are both integers. Their difference is their behaviour.

Ashley Smith00:08:20

Nice! Right now I'm not worrying about the user as im doing a guest post so I'm expecting that to be null

Ashley Smith00:08:06

unfortunately I am still getting my error 😞