Fork me on GitHub
#beginners
<
2022-12-14
>
Abhi Saxena06:12:37

Hi Clojurians, I am trying to store a Date String in Postgres with insert statement, whatever format I try it stores a different value in DB, any clue on which format should I use. e.g. "01/01/2022" ends up with 0, "2014-01-01" stored as 2012. the DB column is of type text (varchar). Do I need to parse the string before storing in DB.

pavlosmelissinos07:12:20

Can you share some code? What have you tried so far?

pavlosmelissinos07:12:56

Also what libraries are you using, etc? Postgres has a Date type, it's better to use that rather than strings to represent dates

pavlosmelissinos07:12:20

I'm not sure why you get that behaviour with the strings though

Abhi Saxena07:12:26

If I make it Date type in Postgres it works fine, but I don't have an option to change the data type in DB as that column can store any arbitrary value

Abhi Saxena07:12:04

Some where the String is being parsed, mostly by Postgres

pavlosmelissinos07:12:37

It's a string, so I'm surprised that any parsing is happening given that the destination column is also a string. I won't be able to help you without a reproducible example but you might get a better answer at #sql from more knowledgeable folks (or even here, if you wait a bit) 🙂

Miķelis Vindavs11:12:29

If the column type is string, you need to format the date as a string in the correct format before passing it to the db code. Otherwise the default stringification will be used which might not do what you want

Abhi Saxena22:12:34

thanks @U89SBUQ4T - Do you know which function I can use to format the Date as String.

slk50011:12:41

'the joy of clojure 2nd edition' "Grabbing the first element in a lazy seq built with rest causes a realization as expected. But the same doesn’t happen for a seq built with next because it’s already been previously realized. Using next causes a lazy seq to be one element less lazy, which might not be desired if the cost of realization is expensive. In general, we rec- ommend that you use next unless you’re specifically trying to write code to be as lazy as possible." and after statement above next example using 'rest' not 'next': (def fifth (comp first rest rest rest rest)) (fifth [1 2 3 4 5]) ;=> 5 so is there any reason to use 'rest' here? why no 'next' as they recommend?

Miķelis Vindavs11:12:11

I don’t know about the book, perhaps that’s just an oversight. But using next is usually more convenient because you can check (if coll instead of (if (seq coll)

Miķelis Vindavs11:12:13

e.g. using the (simplified) definition of conj from cljs.core:

(defn conj [coll x & xs]
  (if xs
    (recur (conj coll x) (first xs) (next xs))
    (conj coll x)))

;; vs

(defn conj [coll x & xs]
  (if (seq xs) ;; forgetting the `seq` here could lead to infinite recursion
    (recur (conj coll x) (first xs) (rest xs))
    (conj coll x)))

Miķelis Vindavs11:12:08

in practice, i don’t think i ever use rest

slk50011:12:20

that's good point. I think that in every clojure book I've read examples are with 'rest' not 'next'

Miķelis Vindavs11:12:02

It’s a better name than next imo, maybe that’s why

slk50011:12:51

agree, the name is better

slk50011:12:00

thank you @U89SBUQ4T for explanation & great example, have a nice day!

1
dumrat11:12:46

Quick question: I'm using hiccup, and I want to do something like this:

[:div {:_ "hyperscript script"} ...]
; Or,
[:div {"_" "hyperscript script"} ...]
But hiccup converts both these into
<div true="hyperscript script" ... />
How do I make hiccup behave nice here?

dumrat11:12:50

I'm time constrained so for now I'll just string replace the result I guess zzz

Miķelis Vindavs12:12:27

what are you expecting as the result?

Miķelis Vindavs12:12:07

_ is not a valid html attribute

dumrat13:12:55

I'm expecting <div _="hyper... And hyperscript makes use of this attribute.

skylize14:12:59

I'm not sure what hiccup is doing here. But from the HTML side of things, it is bad practice to use arbitrary attribute names. Custom attributes should be under data-*, which then shows up as an entry in the dataset property of the DOM element. https://developer.mozilla.org/en-US/docs/Learn/HTML/Howto/Use_data_attributes So your example would be [:div {:data-_ "hyperscript script"} ...].

👍 1
dumrat14:12:26

@U90R0EPHA But but hyperscript uses that tag. I can't change it 😞

Rupert (All Street)14:12:34

That's odd - we use hyperscript with hiccup and it works fine. Maybe a different version of hiccup or some post processing that you are doing?

[:a {:_ "on click trigger closeM"} "Close"]

👍 1
dumrat14:12:21

Now it works fine. I swear it was generating the above before. I don't understand.

👍 1
zakkor11:12:40

How can I figure out this error?

; Error printing return value (NullPointerException) at null (REPL:1).
; null
clj꞉aoc2022.aoc2022꞉> 
clojure.main/repl (main.clj:442)
clojure.main/repl (main.clj:458)
clojure.main/repl (main.clj:368)
nrepl.middleware.interruptible-eval/evaluate (interruptible_eval.clj:84)
nrepl.middleware.interruptible-eval/evaluate (interruptible_eval.clj:56)
nrepl.middleware.interruptible-eval/interruptible-eval (interruptible_eval.clj:152)
nrepl.middleware.session/session-exec (session.clj:218)
nrepl.middleware.session/session-exec (session.clj:217)
java.lang.Thread/run (Thread.java:1589)

Rupert (All Street)14:12:45

I guess we would need a bit more context. Most of what we can see from that is a null poionter exception, possibly triggered outside of a source code file (e.g. generated code/repl).

zakkor16:12:23

@UJVEQPAKS well, how can I figure out which context to show you? It doesn't say where the error happened 😄 I created this error by using option+Enter to send this form to REPL using Calva

zakkor16:12:01

(do
    (def d7
      (let
       [fs-zip
        (fn [root]
          (z/zipper
           :children
           :children
           (fn [node children] (assoc node :children children))
           root))

        commands
        (->> (split (slurp "input7") #"\$ ")
             (map split-lines)
             (drop 2))

        parse-file-entry
        (fn [s] (let [[size-or-type name] (split s #" ")]
                  (if (= size-or-type "dir")
                    {:name name :children []}
                    {:name name :size (parse-long size-or-type)})))

        zip-find
        (fn [loc pred]
          (if (pred (z/node loc))
            loc
            (if (z/end? loc)
              nil
              (recur (z/next loc) pred))))

        interpret-cmd
        (fn [loc [cmd & output]]
          (if (= cmd "ls")
            (z/edit loc #(assoc % :children (map parse-file-entry output)))
            (let [[_ arg] (str/split cmd #" ")]
              (if (= arg "..")
                (z/up loc)
                (zip-find loc (fn [node] (= arg (:name node))))))))

        node-size
        (fn node-size [node]
          (if (:size node)
            (:size node)
            (reduce + (map node-size (:children node)))))]
        
        (->> commands
             (reduce interpret-cmd (fs-zip {:name "/" :children []}))
             z/root
             fs-zip
             zip-iter
             (map (fn [loc]
                    (let [node (z/node loc)]
                      (if (:children node)
                        (assoc node :size (node-size node))
                        node))))
             (filter #(and (:children %) (< (% :size) 100000)))
             (map :size)
             (reduce +))
        #_(reduce interpret-cmd (fs-zip {:name "/" :children []}) commands)
        #_commands
        ))
    d7)

Rupert (All Street)16:12:28

Try sending small bits of it to the repl to see which bit breaks it.

zakkor16:12:43

I could do that, or use a debugger, but I feel like it's unacceptable for an error to not even point to the actual line where the error happened? It's not possible to fix this?

Rupert (All Street)16:12:54

A few things: • Make sure your execution is working (e,.g. make sure (+ 1 2) runs correctly.) • Try turning your code info a function and then evaluating it. • If you are pasting into a REPL instead of a using a namespace code - try using a clojure file with a namespace. • Add log statements • You have lots of anonymous functions that you can add names to to make it easier to see if the exception is happening inside of them.

(fn you-can-write-anything-here [root]
          (:zipper
           :children
           :children
           (fn [node children] (assoc node :children children))
           root))

zakkor16:12:17

Ah! I found it

zakkor16:12:19

aoc2022.aoc2022/eval9913 (NO_SOURCE_FILE:245)

zakkor16:12:46

I only manged to get this actual line in a different error though, the original one really does not say anything useful

zakkor16:12:06

Also, log statements don't work because the output doesn't get flushed it seems

Rupert (All Street)16:12:56

I would highly encourage you to fix logging - it is much easier to make progress with them working.

Rupert (All Street)16:12:37

In general log lines are always flushed immediately in most frameworks (since devs like to see them straight away).

zakkor16:12:25

well I was just using println lol

Rupert (All Street)16:12:38

Yeah that should work

Rupert (All Street)16:12:18

if the println isn't printing it probably means the line hasn't been evaluated or that *out* or System/out has been overriden.

phill23:12:28

Sometimes an inscrutable stack trace like this is all you get when the problem occurs when Clojure is working its way through a lazy sequence that your code set up. You can surround calls to lazy functions such as map with doall to force the sequence to be realized then-and-there and if an exception occurs you'll recognize the context. Another thing that helps is to insert println between the pairs in a let, if you suspect the problem is in the let somewhere but you don't know where. How do you do that? you include a pointless assignment, such as " (println ...)", in the let. The nil that's returned by println is harmlessly bound to the variable .

Miguel13:12:45

Anyone using https://github.com/miner/clj-ftp for storing files in an FTP server? I have a working version but it requires storing the whole file at once, but I would rather append to the file inistead. Since clj-ftp does not support append, I wrote my own function but If I don’t close the writer before appending the file it just hangs and nothing happens. And if I do close the writer, on the next iteration of doseq the stream is closed

(comment

  (defn client-append-stream
    "Put an InputStream (must be within a with-ftp)"
    [client instream remote]
    (.appendFile ^FTPClient client ^String remote ^InputStream instream))

  (let [out       (PipedOutputStream.)
        in        (PipedInputStream. out (* 256 1024))
        writer    (io/writer out)
        file-name "test.csv" 
        url       "myftpserver"
        ftp-path  ["folder"]]
    (doseq [i [1 2 3]]
      (.write writer "abc")
      (.flush writer)
      ;(.close writer) ; Must close here but then exception about stream closed is thrown
      (ftp/with-ftp
        [client url
         :data-timeout-ms 10000, :control-keep-alive-timeout-sec 10
         :control-keep-alive-reply-timeout-ms 500]
        (run! (fn [p] (ftp/client-cd client p)) ftp-path)
        (client-append-stream client in file-name)
        (ftp/client-all-names client)))
    (.close writer)
    (.close in)
    (.close out))
  )

Rupert (All Street)14:12:45

Maybe move the let inside of the doseq so you get a fresh writer each time. Generally when an object is closed it can't be unclosed. If you don't care about an exception you can wrap it in (try..catch) and ignore it. Typically when you close a writer it will also recursively call close on underlying closeable objects - so you may have to reopen fresh objects each time. As you may know, If you're not using a VPN or SSH tunel -then FTP traffic travels unencrypted (including the pasword to the FTP server) , so do consider SFTP as an alternative if needed.

Miguel14:12:59

That seems to work, thanks, but why does the writer need to be closed to store the files in FTP? Shouldn’t flush be enough?

👍 1
Rupert (All Street)14:12:39

Usually - but it depends on the implementation.

🙏 1
radu14:12:39

What's the nicest way to re-write the following in Clojure?

var foo = getFooFromSomewhere();
if (foo == null) {
    foo = getFooFromSomewhereElse();
}
if (foo == null) {
    foo = getFooFromYetSomewereElse();
}
if (foo == null) {
    return null;
}

return computeBaz(foo);

ghadi14:12:31

(or (alt1) (alt2) (alt3) (fallback ...))

👍 3
radu15:12:43

nice, thanks Ghadi

bschrag14:12:36

Any snappy idioms for counting the number of characters consumed by read-string---particularly when returned value is an s-expression? Or any other snappy way (considering optional white space) to advance my cursor to read a next s-expression? My input includes plain symbols and vectors with embedded arbitrary Clojure forms. Must I really traverse the input string, counting and balancing opening and closing brackets? I feel like I'd be duplicating the reader. (Common Lisp's read-string conveniently returns the number of characters read, as a second value. I suppose there's some deep reason Clojure doesn't, at least optionally...)

Rupert (All Street)14:12:20

You can switch to clojure.edn/read or clojure.core/read as these take java.io.PushbackReader instead of a string - You can create your own version of it using proxy (wrapping/delegating to a regular java.io.PushbackReader)- then you can put counters etc inside your implementation. This will only help for certain situations though.

Alex Miller (Clojure team)14:12:43

Use read+string to give you read string back as well

Rupert (All Street)14:12:47

If you don't need an exact value - you can estimate it from the resulting data. Take the output datastructure and call (count (pr-str datastrcuture)) .

bschrag14:12:29

@U064X3EF3 How can I "switch?"

(read+string "foo")
...obviously doesn't work. BTW, I don't have a Java background (getting the sense that I may need to acquire some).

bschrag14:12:49

@UJVEQPAKS I do need an exact value.

Rupert (All Street)14:12:15

I guess Alex was referring to the fact that read-string can be implemented from using the lower level read function. I'm afraid a little Java would possibly help if you are implementing a pushback reader..

Rupert (All Street)14:12:20

Maybe split the string first e.g. line-seq or str/split then run read-string on these smaller strings which are easier to read/count. Depending on your usecase you may end up having to write your own simple parser (adding up brackets etc.

bschrag14:12:05

"Pushback" here refers to the remaining string (or something that you can use to get it)?

Rupert (All Street)14:12:03

A pushback reader wraps around the string - it's like a string - but with a position and also the ability to "unread data" if your position has gone to far.

Rupert (All Street)14:12:20

What are you trying to count?

hiredman14:12:53

Why does read+string obviously not work?

hiredman14:12:32

read+string returns a pair of the value produced by reading, and the string that was read

hiredman14:12:40

The string that was read can be counted

bschrag14:12:47

@U0NCTKEV8

template-matcher.core> (read+string "foo")
Execution error (ClassCastException) at template-matcher.core/eval6838 (form-init14808213374948926050.clj:266).
class java.lang.String cannot be cast to class clojure.lang.LineNumberingPushbackReader (java.lang.String is in module java.base of loader 'bootstrap'; clojure.lang.LineNumberingPushbackReader is in unnamed module of loader 'app')

Rupert (All Street)15:12:45

Actually - what you can do is use Pushback reader + read. Once you have called read you can turn what remains on the pusback reader back into a string. Then your count (- (count original-text) (count text-left-on-pushbackreader) . Maybe this is what Alex was suggesting.

☝️ 1
bschrag15:12:15

@UJVEQPAKS I've been interested in counting because I supposed a Common Lisp-like approach could be appropriate. If a pushback reader can give me the remaining string directly, that'd be all that I'd need.

Rupert (All Street)15:12:49

I think that might be the way to go then. You just need to work out how to go from String -> java.io.PushbackReader (and possibly the reverse). I don't have time to look it up right now. But if you're still stuck later I can check.

Ed15:12:54

if you just want a hacky answer, you could always do (read-string (str "[" "foo bar baz" "]"))

Ed15:12:50

if you want a pushback reader from a string, you need to construct a string reader first ...

(let [r (java.io.PushbackReader. (java.io.StringReader. "foo bar baz"))]
    (take-while (complement #{::eof}) (repeatedly #(read r false ::eof))))

bschrag15:12:56

@U0P0TMEFJ Trying to read successive s-expressions from a string... Ah, some pushback reader fu!

bschrag15:12:53

> (let [r (java.io.PushbackReader. (java.io.StringReader. "(this is) a    spacey   [test]"))]
                         (take-while (complement #{::eof}) (repeatedly #(read r false ::eof))))
((this is) a spacey [test])
Very nice. 🙂 Thanks, everyone!

👍 1
stantheman16:12:58

You seem to get lots of warnings about clojure.core/read as it is allowed full eval but it would translate to clojure.edn/read ...

👍 1
skylize14:12:16

Is it possible to embed a quick-check test in the :test meta of a function. 🧵

skylize14:12:03

(require '[clojure.test.check :as tc]
         '[clojure.test.check.generators :as gen]
         '[clojure.test.check.properties :as prop])

(defn foo
  "An empty fn with a property test that should always fail."
  {:test (fn []
           (let [prop (prop/for-all [_ (gen/return nil)]
                        (= true false))]
             (tc/quick-check 1 prop)))}
  [])

(defn bar
  "An empty fn with a property test that should always pass."
  {:test (fn []
           (let [prop (prop/for-all [_ (gen/return nil)]
                        (= true true))]
             (tc/quick-check 1 prop)))}
  [])

(t/run-tests)
; => {:error 0, :fail 0, :pass 0, :test 2, :type :summary}
;  😞

lassemaatta15:12:21

I might be wrong, but doesn't tc/quick-check just return the test results?

skylize15:12:34

Yes. That sounds about right. But I don't what else I would use here. I'm generally pretty confused about the whole global state situation of clojure testing. So far, I have just used defspec for making prop tests runnable with a test suite.

lassemaatta15:12:13

What I meant was that what if you add something like (let [results (tc/quick-check 1 prop)] (when-not (:pass? results) (throw (ex-info "fail" results))

skylize16:12:12

Or, I guess I need to stick that in an is?

lassemaatta16:12:46

sure, that might (also) work

skylize19:12:37

So I got hooked into clojure.test using is like so:

(if (:pass? test-result) (t/is true)
                 (throw (ex-info "foo test failed" test-result)))
But somehow this just feels wrong to use is to report a pass, while all the conditional logic is outside of it. Is defspec the only method that test.check offers for hooking directly into clojure.test?

skylize14:12:33

Found the solution in the source:, a check-results assertion.

(require '[clojure.test.check.clojure-test.assertions :as assert])
...
 {:test (fn []
           (let [prop (prop/for-all [_ (gen/return nil)]
                        (= true false))]
             (assert/check-results
              (tc/quick-check 5 prop))))}
...
; =>
FAIL in (foo)
expected: {:result true}
  actual: {:shrunk {:total-nodes-visited 0, :depth 0, ...
...
  {:test (fn []
           (let [prop (prop/for-all [_ (gen/return nil)]
                        (= true true))]
             (assert/check-results
              (tc/quick-check 5 prop)))}
...
; =>
{:error 0, :fail 0, :pass 1, :test 1, :type :summary}

👍 1
dumrat15:12:55

I was doing a company hackathon to make a simple responsive support tool with clojure + htmx. One thing that keeps coming up with Clojure came up here too. But I was pressed for time so this time I really felt it. It's to do with changing shape of data. I'd have some assertions about a piece of data in some place and then the code would evolve so that I'll change the shape somewhere else and then I get really weird errors and spend a lot of time fighting this. I've seen it suggested somewhere that you should always spec your data. Perhaps I'll give that a try. Aside: Is it a good idea to use malli/spec validators on atoms? Any general advise for this problem?

Rupert (All Street)15:12:26

There is a quote: > "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." —Alan Perlis Practical things I take away from this: • Get to know the data structure - really be able to visualise it in your head (/print it to REPL/explore it) • Don't constantly change it (e.g. if it starts as a sequence of maps keep it as a sequence of maps) instead of changing it over and over again so it's hard to keep up with how the data looks at any given stage. ◦ Do your data transformation once up front into your ideal structure and then let the rest of your code work just with that ideal structure (not lots of different ideal structures). If in doubt take a look at the data (use logging, debugger or even an inline def that you can investigate in the REPL): (do (def a data) (map inc data)) is valid (especially during development).

👍 1
practicalli-johnny15:12:03

My initial thoughts are that the issues are because of shared state, i.e the atom that is used. For the web UI work I do, I create a single data structure (only using an atom if there are dynamic changes) and send only the relevant parts of that structure to functions that will process/ transform that data. This reduces the impact of change A simple example is in https://github.com/practicalli/practicalli.github.io/blob/live/src/practicalli/landing_page.cljs Which keeps all the data in a practicalli.data namespace and sends just the parts it need to the relevant functions I tend to keep clojure.spec focused on data coming into the system (e.g. API calls) or outgoing (database) or both (Kafka, message queues). For internal data, the Repl is enough for me to understand if I am using the right data

👍 1
phill23:12:03

Another technique is not to change the shape. Instead, make a new shape, operated on by a new, parallel set of functions (typically you make them quickly by copy-and-paste from the original). If the program is larger than your brain, you attack 1 portion of your program at a time, and at the edge of that portion just convert the "old" data to the "new" data on the way in, and reverse on the way out. A side benefit is that (insofar as your functions are pure) a shim for the new version of function x can run both the new and old versions of x and assert that they return the same (or equivalent) thing.

👍 1
rjray17:12:37

I've gotten rusty at Clojure-- I seem to remember a keyword that would take a seq like (1 2 3 4) and return adjacent pairs? I.e., ((1 2) (2 3) (3 4)). Ring any bells? partition doesn't overlap.

hiredman17:12:34

partition can overlap

hiredman17:12:58

user=> (doc partition)
-------------------------
clojure.core/partition
([n coll] [n step coll] [n step pad coll])
  Returns a lazy sequence of lists of n items each, at offsets step
  apart. If step is not supplied, defaults to n, i.e. the partitions
  do not overlap. If a pad collection is supplied, use its elements as
  necessary to complete last partition upto n items. In case there are
  not enough padding elements, return a partition with less than n items.
nil
user=>

hiredman17:12:39

you may want to check out partition-all as well (although that can be less of issue when overlapping)

rjray17:12:02

Ah! I knew there was a command, I just forgot that partition can take a step value. Thanks!

Falak Shair20:12:14

How to resolve this error? myapp > lein run Exception in thread "main" java.lang.ExceptionInInitializerError at clojure.main.<clinit>(main.java:20) Caused by: Syntax error compiling deftype* at (flatland/ordered/set.clj:19:1) . . . Caused by: java.lang.IllegalArgumentException: Must hint overloaded method: toArray at clojure.lang.Compiler$NewInstanceMethod.parse(Compiler.java:8496) lein version Leiningen 2.10.0 on Java 17.0.5 OpenJDK 64-Bit Server VM

Martin Půda20:12:47

Probably missing type hints. Can you show us your code (or, if it's long, link to the repository)?

seancorfield20:12:17

Looks like @U04AP46SGDU has an older version of flatland/ordered since line 19 of set.clj is very different in the updated library. If you're not using it directly, you may have an old version of some other dependency that is bringing that in. But updating to more recent versions of libraries should fix that problem since it was fixed over two years ago in ordered.