Fork me on GitHub
#beginners
<
2022-10-11
>
Abhi Saxena05:10:13

Hi All, I am using HoneySQL for upsert command with Clojure.java.jdbc, the connection object looks like this: db-spec {:pool #object[clojure.lang.Delay 0x4669f730 {:status :pending, :val nil}], :options {:naming {:keys #object[clojure.core$identity 0x4f6dfd28 "clojure.core$identity@4f6dfd28"], :fields #object[clojure.core$identity 0x4f6dfd28 "clojure.core$identity@4f6dfd28"]}, :delimiters ["\"" "\""], :alias-delimiter " AS ", :subprotocol "postgresql"}} is missing a required parameter It keeps complaining that it's missing a required parameter, how do I know which one is missing. The function looks something like this:

(defn insert-client-targets [context study-id database-id arguments]
  (let [table_key "it_client_unq"]
    (jdbc/execute! (get-db context) (-> (honey/insert-into table_name)
                                        (honey/values [{:client_id         (get-in context [:context :client-id])
                                                        :database_id       database-id
                                                        :study_id          study-id
                                                        :target_loss     (:target_loss arguments)}])
                                        (honey/upsert (-> (honey/on-constraint table_key)
                                                          (honey/do-update-set (generate-update-input-map arguments (atom {})))))
                                        (honey/returning :*)))))

dumrat06:10:14

Can you print out the sql generated by your thread first expression? Is that valid?

hiredman06:10:56

You have a delay in the middle of your connection spec

hiredman06:10:38

Like, it has nothing to do with honeysql, your db spec is just bogus

👍 1
Abhi Saxena06:10:14

Here is the sql - {:rel {}, :pk :id, :table table_name, :db {:pool #object[clojure.lang.Delay 0xb42c012 {:status :pending, :val nil}], :options {:naming {:keys #object[clojure.core$identity 0x4f6dfd28 clojure.core$identity@4f6dfd28], :fields #object[clojure.core$identity 0x4f6dfd28 clojure.core$identity@4f6dfd28]}, :delimiters [" "], :alias-delimiter AS , :subprotocol postgresql}}, :name table_name, :do-update-set [{:target_loss 440}], :transforms (), :fields [], :returning [:*], :on-constraint it_client_unq, :values [{:client_id 400, :database_id 909, :study_id 101, :target_loss 440}], :insert-into [], :prepares ()}

Abhi Saxena13:10:27

I was using the DB connection for Korma library, I replaced it now with below config, still same error - Here is the output of conn object - {:subprotocol postgresql, :subname //localhost:5432/xyz_app?currentSchema=public, :user username, :password password, :classname org.postgresql.Driver, :make-pool? true, :maximum-pool-size 100, :minimum-pool-size 30}

Abhi Saxena13:10:52

db-spec org.postgresql.jdbc.PgConnection@5bba441c is missing a required parameter

rolt15:10:27

you're passing the connection object, it's expecting the db-spec {:subprotocol xxx :subname xxx ...} you could also pass the connection but then you should wrap it like that: {:connection ConnObject} hard to give more info without seeing the actual code, but i hope this helps you identifying the problem

Abhi Saxena15:10:32

Here is the code I have:

(with-open [conn (jdbc/get-connection it-app-db)]
  (jdbc/execute! conn (-> (honey/insert-into :target_table)
                          (honey/values [{:client_id         (get-in context [:context :client-id])
                                          :database_id       database-id
                                          :study_id          study-id
                                          :target_loss       (:target_loss arguments)}])))(.close conn))

Abhi Saxena15:10:29

Here is my it-app-db {:subprotocol postgresql, :subname //localhost:5432/it_app?currentSchema=public, :user username, :password password, :classname org.postgresql.Driver, :make-pool? true, :maximum-pool-size 100, :minimum-pool-size 30}

Abhi Saxena16:10:32

@U02F0C62TC1 - thanks Roland, I was able to make it work with db-spec

Michaël Salihi14:10:59

Hi, I wonder to know if there is a better approach to check if collection contains nil or false values:

(some #(or (nil? %) (false? %)) [nil true false])

Ben Sless14:10:18

Why not use identity ?

👍 1
Chris G14:10:04

you could probably do some-fn

👍 1
Michaël Salihi14:10:53

Mmm interesting, so with some-fn:

(apply (some-fn nil? false?) [true nil false])

delaguardo14:10:37

(reduce #(when-not %2 (reduced true)) [1 false nil 4 5])

Martin Půda14:10:15

(some #(not %) [nil true false])

💯 2
Michaël Salihi14:10:13

Thanks you all for these suggestions! Really like not solution from @U01RL1YV4P7!

elken15:10:17

Don't even need the lambda

Michaël Salihi15:10:32

We can even rewrite it like this:

(some not [true false nil])

Michaël Salihi15:10:36

Haha we're in sync @U043RSZ25HQ 🙂

elken15:10:32

Great minds think alike 😉

😉 1
dumrat15:10:09

Hi, why would this return nil to me? The println actually does print the entire response. Note that this is in SCI

(let [data (atom "")]
    (https/get "" #js {}
               (fn [^js res]
                 (.on res "data" (fn [chunk]
                                   ;(println (.toString chunk))
                                   (swap! data str (.toString chunk))))))
    @data)

borkdude15:10:29

because async?

borkdude15:10:57

which SCI environment is this?

borkdude15:10:30

but is this #C029PTWD3HR or ...?

borkdude15:10:48

oh joyride?

borkdude15:10:51

yeah of course

skylize17:10:40

That's classic JS async problem. You are calling @data immediately after submitting get request. Node includes a util to convert callback api to promise api, so you can then use with Promesa. (They also added some pre-promisfied versions of many fns. So you might want to search if I missed this one.)

(def get (util/promisify https/get)
(-> (get url )   ;; <-- or (get opts) or (get url opts)
  (p/chain handle-resp)

👍 1
Joe Duhamel15:10:35

is there a reasonable way in the repl to inspect what was in deps.edn?

Alex Miller (Clojure team)15:10:25

probably, but what specific question are you trying to answer?

Joe Duhamel15:10:53

really wanted to look at paths/aliases and deps. just making sure stuff was getting pulled in.

Alex Miller (Clojure team)15:10:58

(require '[ :as jio] '[clojure.edn :as edn])
(def basis (-> (System/getProperty "clojure.basis") jio/file slurp edn/read-string))

Joe Duhamel15:10:11

excellent, thanks I'm a old (too old) common-lisp guy moving into clojure.

Alex Miller (Clojure team)15:10:17

will give you the basis that was used to start the repl. the basis is a superset of the merged deps.edn files so will have a lot of the data in it

Alex Miller (Clojure team)15:10:24

(pprint (:classpath-roots basis))
might be helpful for example

Joe Duhamel15:10:25

was perfect, I was trying to understand how the merging was done.

Abhi Saxena18:10:22

Hi All, How do I make sure that the form from else block in below if statement returns, It always returning nil :

(defn handle-put-client-targets [context code-id area-id arguments]
  (let [forecast-info (get-forecast-info context code-id area-id)]
    (if forecast-info (throw (ex-info (format "no forecast attached for area id %d and code id %d"
                                              code-id area-id) {:cause :bad-request}))
                      (upsert-client-target context code-id area-id arguments))))

dpsutton19:10:59

what does upsert-client-target return?

dpsutton19:10:34

are you positive? Seems like that function should either throw or return upsert-client-target ’s return. And if you are getting nil it means we don’t understand something. • is the function actually getting called? • is an error thrown and it’s being handled elsewhere? • upsert-client-target can return a nil in some cases?

Abhi Saxena19:10:35

but whenever I am calling handle-put-client-targets, even though upsert-client-target is executed, I get back NIL

Abhi Saxena19:10:33

• is the function actually getting called? - Yes, its been called • is an error thrown and it’s being handled elsewhere? - No exception thrown • upsert-client-target can return a nil in some cases? No

dpsutton19:10:34

Not sure how to help more with this code fragment then. Sorry

Abhi Saxena19:10:09

thanks Dan for your prompt response

dpsutton19:10:26

no worries. Things that can help. Are you able to call the functions from the repl (no webserver and such involved?) when this happens is forecast-info present or not?

Abhi Saxena19:10:08

forecast info is not present and it does call Upsert-client-target, but somehow it doesn't return the record updated.

dpsutton19:10:16

can you call upsert-client-target with the same arguments from the repl? What happens? Can you call it twice in a row with the same values, etc

Abhi Saxena19:10:09

yes, in both cases it works

dpsutton19:10:32

then that makes me think the execution flow is not what you suspect it is

Chris G19:10:58

why do you want to throw if the forcast-info binding is true?

Chris G19:10:19

is that the intention?

Chris G19:10:02

seems like you'd want if-not

Abhi Saxena19:10:13

Okay, thanks Chris I will try with if-not

vlad_poh19:10:31

If i have a JWT that expires after 5 minutes and a refresh JWT that expires after 90. What is the best/right/idiomatic approach to refreshing the token in a simple clojure app/service? I’m not sure adding a watcher, using a future or core.async are my only options. And if they are which would be the least brittle answer to my problem.

hiredman19:10:36

I have written a few of these little loops, usually using core.async that maintain a valid token and provide it over a channel to any code that asks for it

hiredman19:10:30

But I haven't bothered to abstract out the common parts between the different API clients I've written in our code base at work

hiredman19:10:15

I think they often end up with a kind of tortured nested use of alts!, which is something to watch out for

vlad_poh20:10:52

@U0NCTKEV8 can you elaborate on how you use core.async? Are you looping every minute and checking the token’s validity?

hiredman20:10:49

Depends the loops for different clients are slightly different

hiredman20:10:50

I guess I don't ever check a tokens validity. The code gets a token some how, and tokens have a known lifetime, and at something like 80% of the lifetime I get a new tolen

👍 1
hiredman20:10:26

the loops look something like

(fn f [ch stop get-token]
  (go
  (alt! stop
        ([_] nil)
        (get-token)
        ([t]
         (let [to (timeout ...)]
            ((fn g []
              (go
              (alt! stop
                  ([_])
                  [[ch token]]
                  ([given?]
                    (when (some? given?) (g)))
                  to
                  ([_]
                    (f)))))))))))

Eric19:10:18

I’m working through some clojure exercises. Anyone know a better way to write this function? Problem: Write the function validate that checks whether a map contains the correct keys and data types. The function expects a map of required keys and predicates and a map of general data that will be validated.

(defn validate
  [validators record]
  (and 
    ;; check if all keys are present
    (reduce (fn [bool k] (and bool (contains? record k))) 
            true 
            (keys validators))
    ;; check if all predicates return true when applied to their mapped value
    (reduce (fn [bool k] (and bool ((k validators) (k record)))) 
              true 
              (keys validators))))
;;Examples:
(validate {:a int? :b string? :c int?} {:a 1 :b "hi" :c 3 :d 20.5}) ; => true: all keys present and all predicates check out
(validate {:a int? :zzz int?} {:a 1 :b "hi" :c 3 :d 20.5}) ; => false: not all keys present
(validate {:a int? :b int?} {:a 1 :b "hi" :c 3}) ; => false: not all predicates check out
This works, but it doesn’t have a way to stop early if it encounters a missing key or a false predicate. Also it feels more verbose than it needs to be.

phronmophobic19:10:53

Here's some hints: • check out some of the functions from https://clojure.org/reference/sequences#_using_a_seq • reduce supports early termination via reduced

(reduce (fn [sum num]
          (if (> num 10)
            (reduced sum)
            (+ sum num)))
        (range))

Eric19:10:49

Thanks for the tips. Nice. It looks like every? will do the job. I’ll try that

🎉 2
Eric19:10:31

There was a (badly-named?) macro in Racket that was called andmap that worked like that. So I guess I just didn’t know what to look for in this case.

skylize22:10:16

This sounds strikingly similar to your question yesterday. You can re-use that work, and then just check the resulting collection for anything non-true.

;; `apply-same-key-to` is what you wrote yesterday,
;; minus the outer part that maps it over a list of records

(let [validated (apply-same-key-to validators record)]
  (not (some #(not= true %) validated))
https://clojurians.slack.com/archives/C053AK3F9/p1665439522362959?thread_ts=1665439522.362959&amp;cid=C053AK3F9

Abhinav07:10:34

every? is more idiomatic for this use case. rather than reduce.

(defn validate
  [validators record]
  (every? (fn [[k v]] ; destructuring the map
            (and (contains? record k)
                 (contains? validators k)
                 ((k validators) v)))
          record))


(validate {:a int? :b string? :c int?} {:a 1 :b "hi" :c 3}) ; true
(validate {:a int? :zzz int?} {:a 1 :b "hi" :c 3 :d 20.5}) ; false
(validate {:a int? :b int?} {:a 1 :b "hi" :c 3}) ; false

sheluchin20:10:09

How common is it for multiple namespaces to exist in one file?

phronmophobic20:10:09

My guess is very, very uncommon

sheluchin20:10:09

I'm trying to model the relationship between files and namespaces into a normalized schema. I thought foo.cljc -> foo (clj) + foo (cljs) would be the right way but it's kind of awkward. If I want to list all of a library's namespaces, doing it this way produces a lot of "false duplicates". I'm thinking of just making a single namespace with a boolean fields to indicate language. Any thoughts on that, @U7RJTCH6J?

hiredman20:10:44

It might be slightly more common to have the inverse, a single namespace definition spread across multiple files

hiredman20:10:29

(clojure.core is spread across a few)

sheluchin20:10:04

@U0NCTKEV8 I never considered that possibility :thinking_face:

hiredman20:10:16

Have you looked at the codeq schema? Codeq is a code analysis tool written by rhickey. I don't think it even bothers with namespaces as a first class thing, just definitions and files

phronmophobic20:10:20

The model is kind of tricky, You might even end up with multiple definitions of namespace. Namespaces might be whatever you end up with in memory if you start a clean repl and require all the files in some source path. However, you can also run (defn foo []) at the repl, which will intern into what ever namespace is in *ns* so it's not necessarily tied to a file. You may get a totally different result using static analysis like clj-kondo. There's also the source files.

phronmophobic20:10:48

I would probably go with:

Namespaces might be whatever you end up with in memory if you start a clean repl and require all the files in some source path.
but leave the model open to other interpretations (ie. allow :clj-kondo/namespace to be a different value)

sheluchin20:10:18

@U0NCTKEV8 I have codeq somewhere in my notes to look at eventually but haven't taken a dive yet. I'll take some time to review it sooner, now that you mention it. It is interesting that it doesn't put namespaces as a first class thing. I've kind of gone under the assumption that they should be, but this rhickey guy you speak of might know better 😏

sheluchin20:10:38

@U7RJTCH6J good point on those potential gotchas/edge cases. I am actually okay with having imperfect data as long as it covers the vast majority of cases well. I am using clj-kondo analysis data as my primary data source and don't plan on doing it otherwise in the foreseeable future. clj-kondo itself does the cljc -> 2 namespace definitions thing I mentioned above, but after using it like this for a while, I'm thinking of joining them and making :clj? :cljs? attributes available to indicate the details.

phronmophobic20:10:01

you may want to make it a open ended property rather than just :clj? and :cljs? since there's multiple flavors of each, :bb, :cljd, :cljr, :jank, etc

phronmophobic20:10:04

If the definition comes from a cljc file, you may want the original source text and the form after read for each relevant reader conditional

sheluchin20:10:58

That is true, there are all those other flavors. For now I just want something that works well enough with clj/s before moving on to the more niche options. Thank you very much for the input. Especially with the codeq stuff, there's much food for thought here.

Abhi Saxena21:10:51

Is there a way to return the last record updated/ inserted with Honeysql and Clojure.java.jdbc, I have return below upsert function and it always returns nil:

(defn upsert-client-targets [context area-code device-id arguments]
    (with-open [conn (jdbc/get-connection it-postgres-db)]
      (jdbc/execute! {:connection conn} (-> (insert-into :pa_client_supplied_study_target)
                                            (values [{:device_id       device-id
                                                      :area_code          area-code
                                                      :target_loss   (:target_loss arguments)}])
                                            (upsert (-> (on-conflict :client_id :database_id :study_id)
                                                              (do-update-set (generate-update-input-map arguments (atom {}))))) (returning :*) sql/format)) (.close conn)))

seancorfield21:10:29

(I strongly recommend using next.jdbc instead of clojure.java.jdbc -- the docs are much, much better for next.jdbc and I am actively maintaining it; I stopped maintaining clojure.java.jdbc quite a while ago and it has only got security patches since then: https://github.com/clojure/java.jdbc/tags )

Abhi Saxena21:10:34

Sure thing Sean

Abhi Saxena22:10:24

Going to jdbc.next is not feasible right now, lot of libraries need to be updated and tests are breaking.I tried with

{:return-keys [:client_id]}
at the end of sql/format and it didn't seem to work. I did update the clojure.java.jdbc to 0.7.12

Abhi Saxena22:10:16

(defn upsert-client-targets [context area-code device-id arguments]
    (with-open [conn (jdbc/get-connection it-postgres-db)]
      (jdbc/execute! {:connection conn} (-> (insert-into :pa_client_supplied_study_target)
                                            (values [{:device_id       device-id
                                                      :area_code          area-code
                                                      :target_loss   (:target_loss arguments)}])
                                            (upsert (-> (on-conflict :client_id :database_id :study_id)
                                                              (do-update-set (generate-update-input-map arguments (atom {}))))) (returning :*) sql/format){:return-keys [:client_id]}) (.close conn)))

Abhi Saxena23:10:39

Finally it worked, thanks Sean for your help!!!

seancorfield23:10:28

Given that you have with-open, you don't need that .close call there BTW.