Fork me on GitHub
#beginners
<
2020-12-01
>
zach01:12:19

Hi, i was wondering if anyone had experience using clj-http’s async requests? I am trying to make a request to an endpoint that is cancelled after 2 seconds. If it succeeds, it returns the response body, if it is cancelled or fails it returns nil.

zach01:12:20

in the [clj-http readme](https://github.com/dakrone/clj-http) there’s a nice example of an async request with cancellation;

(import '(java.util.concurrent TimeoutException TimeUnit))

(let [future (client/get ""
                         {:async true :oncancel #(println "request was cancelled")}
                         #(println :got %) #(println :err %))]
  (try
    (.get future 1 TimeUnit/SECONDS)
    (catch TimeoutException e
      ;; Cancel the request, it's taken too long
      (.cancel future true))))

seancorfield03:12:55

@U011DMQ8DSS I'm pretty sure your callback functions have to return their argument (or a modified version of it). Yours return nil (because println returns nil). You want #(do (println :got %) %) #(do (println :err %) %) -- which is what that issue is suggesting: (fn [_} _) returns its argument.

zach04:12:38

Ah, right! I shall try it! I tried something similar to your do function above, but it returned a condensed apache http response, with no response body. I’ll try it again though with some fresh eyes!

zach01:12:13

but the functions do a println, and I want the response body. When I try to just return the response (by switching out #(println :got %) with (fn [res] res)` I only get an apache.http.BasicHTTPResponse, which doesn’t include the body.

zach01:12:02

My issue seems similar to the issue raised in clj-http here: https://github.com/dakrone/clj-http/issues/512, but the function offered as a solution doesn’t seem to be correct…or like valid clojure: `

zach01:12:09

I hesitated on asking this in the clj-http github repo just cos this ticket and its pr are over a year old, and figured I may not get too quick a response, and is likely an issue with my understanding more than the library itself…So I wanted to see if any folk here had experience with this. Thank you!

st3fan02:12:45

Can anyone recommend a scheduler library (cron-like) that is not abandoned 8 years ago? 😕

st3fan02:12:18

I'm browsing through projects on Clojars and so far I mostly see projects without releases, broken documentation and last change 6+ years ago

st3fan02:12:25

I think I just saw a fork that someone picked up again

st3fan02:12:16

I’m thinking I should just build something around java.util.concurrent.ScheduledExecutorService

3
seancorfield03:12:36

@st3fan Or just use Quartz itself via interop?

st3fan03:12:04

I settled on quartzite .. it it not really a pretty api but it works

seancorfield03:12:51

@st3fan I noticed that the fork you mention has posted a single snapshot to Clojars under Daniel's personal group and then switched to gov.nasa.earthdata and posted another snapshot, but I didn't see any non-snapshot releases from that fork yet. Just FYI.

st3fan04:12:30

yeah messy

st3fan04:12:47

I just grabbed clojurewerkz/quartzite instead

yubrshen04:12:16

I'm learning to use jdbc.next, it seems to me that the following code segment caused the error below:

(let [datasource (jdbc/get-datasource (:db env))]
    (jdbc/execute! datasource ["SELECT 1"])
  ...)
The error:
Execution error (ConnectException) at java.net.PlainSocketImpl/socketConnect (PlainSocketImpl.java:-2).
Connection refused (Connection refused)
I'm supposed to use postgres database with the driver and wrapper in deps.edn
seancorfield/next.jdbc {:mvn/version "1.1.613"}
org.postgresql/postgresql {:mvn/version "42.2.11"}
Here is the configuration of the expected database:
:db {:dbtype "postgres"
      :subprotocol "postgresql"
      :subname "postgres"
      :user "postgres"
      :password "scrapsafe"}
I realized that I have not start the postgres database program (server), I guess that that might be the cause of the error. Could you point me to a tutorial of how to install and start the database server and all the required setup/configuration for the database to get the above code working? I'm following the excellent live coding of https://www.youtube.com/watch?v=J4ggrmvnTIY ( Functional Friday 4 - Building a Web App with Clojure ) Thanks!

seancorfield04:12:39

@yubrshen For dev work, I use Docker to stand up a local PostgreSQL instance.

seancorfield05:12:39

Here's what I use for testing clojure.java.jdbc against PG

docker run -p 5432:5432 --name clojure_test -e POSTGRES_PASSWORD=clojure_test -e POSTGRES_USER=clojure_test -d postgres

seancorfield05:12:15

For testing next.jdbc, I use an Embedded PostgreSQL library (which needs special startup in the application) but the same org.postgresql/postgresql driver (version 42.2.10 right now, against the 12.2.0 embedded database).

seancorfield05:12:32

If you don't want to use Docker, you'll have to follow the documentation on the PostgreSQL website. I've never installed PG locally tho', I've always used Docker or the embedded server.

yubrshen05:12:02

@seancorfield Thanks for the prompt help. I'll try the docker first hoping it would work for the purpose of practice of web server application.

yubrshen06:12:45

@seancorfield Unfortunately, with my Ubuntu in WSL1, docker is quite complicated. I may have to try the "Embedded PostgreSQL library" Could you confirm if this is the link to the library, https://github.com/Bigsy/pg-embedded-clj ? If this is the one, the documentation is very sketchy. If it's possible to show the sample code for setup? Thanks!

seancorfield07:12:16

Switch to WSL2. It's worth the effort of starting over and creating a new dev env. I run all my Clojure stuff on WSL2. I have Docker for Redis, PostgreSQL, and ElasticSearch. I use VS Code with the remote-wsl2 extension (so it lets you have all your code and processes on WSL2 but you still run the editor itself on Windows). Then I use Xlaunch (VcXsrv) so I can run X11 GUI stuff like Reveal from WSL2.

seancorfield07:12:41

As for Embedded PG, look in the next.jdbc repo -- the deps.edn file has the deps for the embedded pg library (NOT the one you linked) and the test fixtures code has the startup code for it.

seancorfield07:12:46

(if you're using a real PG DB, you'd want to create a pooled datasource instead from the db-spec, using HikariCP or c3p0)

yubrshen20:12:01

@seancorfield Thanks for the generous help! I'll try them.

yubrshen06:12:30

@seancorfield Following your advice, adding the following dependencies to deps.edn:

seancorfield/next.jdbc {:mvn/version "1.1.613"}
        org.postgresql/postgresql {:mvn/version "42.2.10"}
        io.zonky.test/embedded-postgres {:mvn/version "1.2.6"}
        io.zonky.test.postgres/embedded-postgres-binaries-linux-amd64 {:mvn/version "12.2.0"}
        io.zonky.test.postgres/embedded-postgres-binaries-windows-amd64 {:mvn/version "12.2.0"}
With the following code:
(ns yubrshen.friendwall
  (:require [ring.adapter.jetty :refer [run-jetty]]
            [compojure.core :refer :all]
            [compojure.route :as route]
            [ring.middleware.defaults :refer [wrap-defaults site-defaults]]
            [rum.core :refer [defc render-static-markup]]
            [config.core :refer [env]]
            [next.jdbc :as jdbc])
  (:import (io.zonky.test.db.postgres.embedded EmbeddedPostgres))
  (:gen-class))

(defonce embedded-pg (EmbeddedPostgres/start)) ;; <- line 12
When I ran it as an application:
sudo clojure -M -m yubrshen.friendwall
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See  for further details.
Syntax error (IllegalStateException) compiling at (yubrshen/friendwall.clj:12:1).
Process [/tmp/embedded-pg/PG-4cfbcaeb13da88ab2f0c925cea29854b/bin/initdb, -A, trust, -U, postgres, -D, /tmp/epg61605004942586187, -E, UTF-8] failed
However, when I execute the code of line 12 in REPL, I got the following error:
1. Caused by java.io.IOException
   error=13, Permission denied

          ProcessImpl.java:   -2  java.lang.ProcessImpl/forkAndExec
          ProcessImpl.java:  340  java.lang.ProcessImpl/<init>
          ProcessImpl.java:  271  java.lang.ProcessImpl/start
       ProcessBuilder.java: 1107  java.lang.ProcessBuilder/start
       ProcessBuilder.java: 1071  java.lang.ProcessBuilder/start
     EmbeddedPostgres.java:  624  io.zonky.test.db.postgres.embedded.EmbeddedPostgres/system
     EmbeddedPostgres.java:  249  io.zonky.test.db.postgres.embedded.EmbeddedPostgres/initdb
     EmbeddedPostgres.java:  156  io.zonky.test.db.postgres.embedded.EmbeddedPostgres/<init>
     EmbeddedPostgres.java:  584  io.zonky.test.db.postgres.embedded.EmbeddedPostgres$Builder/start
     EmbeddedPostgres.java:  478  io.zonky.test.db.postgres.embedded.EmbeddedPostgres/start
                      REPL:   12  yubrshen.friendwall/eval20241
                      REPL:   12  yubrshen.friendwall/eval20241
             Compiler.java: 7177  clojure.lang.Compiler/eval
             Compiler.java: 7132  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj: 1973  clojure.core/with-bindings*
                  core.clj: 1973  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj:  660  clojure.core/apply
                regrow.clj:   20  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  834  java.lang.Thread/run
It seems indeed some kind of access permission problem, as I manually checked:
ls /tmp/embedded-pg/PG-4cfbcaeb13da88ab2f0c925cea29854b/bin/initdb
-rwxr--r-- 1 root root 129K Dec  1 21:10 /tmp/embedded-pg/PG-4cfbcaeb13da88ab2f0c925cea29854b/bin/initdb*
only accessible to root! Really appreciate your help again! (I had to use sudo to run the app, as I found that as web server, without sudo, the web server could be started.) I tried to run manually:
sudo /tmp/embedded-pg/PG-4cfbcaeb13da88ab2f0c925cea29854b/bin/initdb -A trust -U postgres -D /tmp/epg61605004942586187 -E UTF-8
[sudo] password for yshen:
initdb: error: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.
It seems that the above might be the clue. I need to learn how to run the app properly. My environment is Ubuntu 18.04 in WSL1 (My Windows 10 is too old for WSL2.)

seancorfield18:12:09

Could be a WSL1 issue. I've never needed to run a separate process -- just start my Clojure app up -- using a non-privileged user. Never seen the problem you're seeing. Like I said earlier, you might be better off just running PG via Docker, using that command I pasted in -- but I suspect you won't be able to with WSL1. I've been running Insider builds of Windows for years (Fast Ring which is now the Dev Channel) so I always have the latest prerelease of Windows and so it's easy for me to run WSL2 etc.

yubrshen18:12:11

@seancorfield Thanks! It's my fault of having run the app by sudo, which caused initdb executable not accessible. The problem has been resolved by not running by sudo.

yubrshen19:12:11

Now, this problem would be really tough for me to figure out. Here is the code where the last line caused the error:.

(ns yubrshen.friendwall
  (:require [ring.adapter.jetty :refer [run-jetty]]
            [compojure.core :refer :all]
            [compojure.route :as route]
            [ring.middleware.defaults :refer [wrap-defaults site-defaults]]
            [rum.core :refer [defc render-static-markup]]
            [config.core :refer [env]]
            [next.jdbc :as jdbc])
  (:import (io.zonky.test.db.postgres.embedded EmbeddedPostgres))
  (:gen-class))

(defonce embedded-pg (future (EmbeddedPostgres/start)))
(def ^:private datasource (atom nil))
(reset! datasource
        (.getPostgresDatabase ^EmbeddedPostgres @embedded-pg)) ; <- failure here
In the REPL, when evaluating the last expression, I got:
1. Unhandled java.lang.ClassCastException
   class io.zonky.test.db.postgres.embedded.EmbeddedPostgres cannot be cast to
   class java.util.concurrent.Future
   (io.zonky.test.db.postgres.embedded.EmbeddedPostgres is in unnamed module of
   loader 'app'; java.util.concurrent.Future is in module java.base of loader
   'bootstrap')

                  core.clj: 2298  clojure.core/deref-future
                  core.clj: 2320  clojure.core/deref
                  core.clj: 2306  clojure.core/deref
                      REPL:   15  yubrshen.friendwall/eval20660
                      REPL:   14  yubrshen.friendwall/eval20660
             Compiler.java: 7177  clojure.lang.Compiler/eval
             Compiler.java: 7132  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj: 1973  clojure.core/with-bindings*
                  core.clj: 1973  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj:  660  clojure.core/apply
                regrow.clj:   20  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  834  java.lang.Thread/run
While running from command line, I only got the following error trace:
{:clojure.main/message
 "Execution error (NullPointerException) at clojure.main/main (main.java:40).\nnull\n",
 :clojure.main/triage
 {:clojure.error/class java.lang.NullPointerException,
  :clojure.error/line 40,
  :clojure.error/symbol clojure.main/main,
  :clojure.error/source "main.java",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type java.lang.NullPointerException,
    :at [clojure.core$apply invokeStatic "core.clj" 665]}],
  :trace
  [[clojure.core$apply invokeStatic "core.clj" 665]
   [clojure.main$main_opt invokeStatic "main.clj" 514]
   [clojure.main$main_opt invoke "main.clj" 510]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]]}}

/tmp/clojure-18342691853881556298.edn (END)
Really appreciate your help!

yubrshen19:12:21

It seems that I'm still missing something for starting the embedded PG, after evaluating (defonce embedded-pg (future (EmbeddedPostgres/start))) I evaluated

(realized? embedded-pg)
Got the similar error:
1. Unhandled java.lang.ClassCastException
   class io.zonky.test.db.postgres.embedded.EmbeddedPostgres cannot be cast to
   class clojure.lang.IPending
   (io.zonky.test.db.postgres.embedded.EmbeddedPostgres and
   clojure.lang.IPending are in unnamed module of loader 'app')

                  core.clj: 7533  clojure.core/realized?
                  core.clj: 7533  clojure.core/realized?
                      REPL:   14  yubrshen.friendwall/eval20703
                      REPL:   14  yubrshen.friendwall/eval20703
             Compiler.java: 7177  clojure.lang.Compiler/eval
             Compiler.java: 7132  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj: 1973  clojure.core/with-bindings*
                  core.clj: 1973  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj:  660  clojure.core/apply
                regrow.clj:   20  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  834  java.lang.Thread/run
evaluating @embedded-pg got similar error.
1. Unhandled java.lang.ClassCastException
   class io.zonky.test.db.postgres.embedded.EmbeddedPostgres cannot be cast to
   class java.util.concurrent.Future
   (io.zonky.test.db.postgres.embedded.EmbeddedPostgres is in unnamed module of
   loader 'app'; java.util.concurrent.Future is in module java.base of loader
   'bootstrap')

                  core.clj: 2298  clojure.core/deref-future
                  core.clj: 2320  clojure.core/deref
                  core.clj: 2306  clojure.core/deref
                      REPL:    0  yubrshen.friendwall/eval20709
                      REPL:   -1  yubrshen.friendwall/eval20709
             Compiler.java: 7177  clojure.lang.Compiler/eval
             Compiler.java: 7132  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj: 1973  clojure.core/with-bindings*
                  core.clj: 1973  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj:  660  clojure.core/apply
                regrow.clj:   20  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  834  java.lang.Thread/run

yubrshen20:12:03

The following seems working:

(def embedded-pg (EmbeddedPostgres/start))
;; => #'yubrshen.friendwall/embedded-pg
(def datasource (.getPostgresDatabase ^EmbeddedPostgres embedded-pg))
;; => #'yubrshen.friendwall/datasource
(jdbc/execute! datasource ["SELECT 1"])
;; => [{:?column? 1}]

seancorfield20:12:52

That's what I pointed you to in next.jdbc's code. Glad you have it working.

yubrshen20:12:51

I was following the test code, which uses future that seems not working as expected in my environment. Thanks again!

Schmoho08:12:16

I have a really hard time understanding basic JNI interaction, I really hope someone can give me a hint here: I'm trying to load a native library with System/loadLibrary on an Ubuntu 20.04 system, the lib very much appears to be on the library path, but the JVM stubbornly claims it can't find it. I've made sure the JVM has read and execution access.

phronmophobic08:12:17

try (System/loadLibrary "lapack")

zackteo09:12:12

Am doing advent of code but am just wondering why are the first 2 elements produced vectors? Am using {clojure.math.combinatorics]

(take 5 (combo/permuted-combinations data 2))
=> ([1757 1890] [1890 1757] (1757 1750) (1750 1757) (1757 1440))

mbjarland10:12:58

A question about (ns … (:gen-class …, how would you create a “singleton pattern” where you have a static method which returns an instance of the gen-classed class? Specifically I have failed to figure out a working way of actually instantiating the gen-classed class within the file where the class is defined

noisesmith16:12:55

usually all class statics would be put into an atom inside a def in the namespace, that can include an instance of the class itself - every def is executed when the namespace itself is loaded

mbjarland20:12:34

How would I instantiate the class from the gen-classed namespace for the class? Is that supposed to "just work"?

mbjarland20:12:12

And thank you for the reply and pointer

noisesmith20:12:16

the def starts out holding nil, then gets a class instance when the static method is first called, reused afterward - delay / promise can be useful here as they enforce the "only set once" property

noisesmith20:12:49

the class instance would be made via the constructor of course

mbjarland06:12:38

And you are supposed to be able to call the constructor from within the ns where the class is defined? As in (ClassName. Args)? I must have had something else broken with my setup then...

noisesmith07:12:32

I think you need to declare :factory name and then call that factory function

mbjarland11:12:36

ah!! :factory x - had totally missed that in the docs. Thank you for that. Sanity restored.

mbjarland10:12:58

ok the above is still of academic interest, but I think I will solve this by moving the relevant functionality to a java class

juniusfree11:12:30

I complted the Day1 challenge but I want to know if there's a better approach. https://twitter.com/juniusfree/status/1333727767658053632

nice 3
zackteo11:12:37

There is #adventofcode btw

joetague15:12:32

I had a go too:

(ns joetague.aoc2020
  (:require [ :as io]
            [clojure.math.combinatorics :as combo]))

(def expense-report (with-open [rdr (io/reader (io/resource "expense_report.txt"))]
                      (doall (mapv #(Integer/parseInt %) (line-seq rdr)))))

(defn line-item-group-matching-sum [col groupsize sum]
  (filter #(= sum (reduce + %)) (combo/combinations col groupsize)))

;; First star
(reduce * (first (line-item-group-matching-sum expense-report 2 2020)))
;; Second star
(reduce * (first (line-item-group-matching-sum expense-report 3 2020)))

joetague15:12:54

I'm not quite sure about my filter because it returns a list (list ()) so thats why I have to do first before my reduce - will see if I can get rid of that

juniusfree21:12:12

@U0G2T8PDM You can check out #adventofcode. I got some great feedback there https://clojurians.slack.com/archives/C0GLTDB2T/p1606822120031800

thanks3 3
0xclj11:12:24

What is the best way to differentiate only the latest content of the email from the context-content(the previous emails auto-added that make up the thread)? Example:

"text" : "On 1\r\nOn 2\r\n\r\nOn 3\r\n\r\n\r\nOn 4\r\n\r\n\r\nOn 6\r\n\r\nOn Tue, Dec 1, 2020 at 4:41 PM Author <[email protected]>\r\nwrote:\r\n\r\n> L\r\n> M\r\n>\r\n> N\r\n> where O>P>Q!\r\n>\r\n> Onyx is a pokemon\r\n> On\r\n> yx\r\n>\r\n> On Tue, Dec 1, 2020 at 4:39 PM Author <[email protected]>\r\n> wrote:\r\n>\r\n>> L\r\n>> M\r\n>>\r\n>> N\r\n>> where O>P>Q!\r\n>>\r\n>> Onyx is a pokemon\r\n"
If I respond to an email, it adds the previous email content as a context that is expanded when you click [...] on gmail. I want to extract only the latest email content. One pattern I see that can work is using:
(-> notification-text
                                      (str/split #"\r\n\r\nOn ")
                                      (as-> raw (raw 0)))
But this will break anytime the email contains a line starting with On . Perhaps parsing it based on the email-content isn’t the best way. What is the recommended way to go about it?

evocatus12:12:39

could you please explain how namespace name depends on directory structure?

andy.fingerhut13:12:51

Tools like Leiningen and the Clojure CLI tools have the notion of a class path. I believe both of them by default have the directory named "src" in the class path.

andy.fingerhut13:12:31

but they can be configured to have a different directory, or a list of directories, in the class path, e.g. src/main/clojure, although that is uncommon.

andy.fingerhut13:12:35

Some characters that are allowed to be in a namespace, like -, must be turned into _ in the corresponding file name. . separators between parts of a namespace must be another level of directory in the file system.

👍 3
evocatus12:12:52

I'm absolutely lost

manutter5113:12:13

(ns my-app.some.lib.util ,,,)

my-project/
  - src/
    - clj/
      - my_app/
        - some/
          - lib/
            util.clj
It’s pretty straightforward, inside your project directory, you have a source directory src and inside the source directory you have nested directories corresponding to the parts of your namespace. If your project has both clj and cljs files, your src directory will have a clj and cljs subdirectory, but if you’re only working with either clj OR cljs files, then you can put your top level dir my-app directly in the src directory. (EDIT: ignore that last bit if it’s confusing, and just put your .clj files in the clj directory--it works fine and its one less thing to worry about.)

👍 6
manutter5113:12:55

The only other wrinkle is that any dashes in your namespace get mapped to underscores in the directory structure, so my-app in the namespace gets mapped to my_app in the directory name.

Schmoho19:12:45

can I somehow inspect the jvm-opts of my running jvm in the repl?

Alex Miller (Clojure team)19:12:31

(System/getProperties) can show you all of the JVM system properties (but not just the ones that were set externally)

👍 3
Schmoho19:12:22

seems like my next question then is something like is there a way to set :jvm-opts in deps.edn regardless of an alias?

{:path     ["src"],
 :deps
 {org.clojure/clojure #:mvn{:version "1.10.1"}}
 :jvm-opts ["--some-option=some-value"]}
I've been trying this, which doesn't work and been reading this https://clojure.org/reference/deps_and_cli#_prepare_jvm_environment but this seems to only refer to using jvm-opts for an alias

Alex Miller (Clojure team)19:12:50

as an aside, for a jvm opt, you'd want that to be ["-Dsome-option=some-value"]

👍 3
Alex Miller (Clojure team)19:12:17

you can only include them as part of an alias right now (or on the command line with -J)

Schmoho19:12:49

thanks for the info. sucks for me, seems like I'll have to figure out how to configure cider to start a repl in a project using command line tools with an alias then or something

Alex Miller (Clojure team)19:12:56

you are certainly not the first to walk this path, not sure if this channel is the best place for the question - maybe #cider or #clojure would be better

Schmoho19:12:11

seems like it thanks a lot 🙂

dpsutton19:12:00

That issue is closed with two solutions listed. What problems are you having?

Schmoho19:12:54

none so far, just thought I'd document this for some reason - not that it's likely someone would be stumbling across this here instead of by googling

dpsutton19:12:57

yeah and this will disappear in five days or so

dpsutton19:12:32

@d.eltzner why do you need an alias though? CIDER does a pretty good job of being able to start up automatically

Schmoho19:12:43

to specify :jvm-opts (see above, apparently that is only possible via aliases)

alex19:12:06

Do folks have any general tips for handling large csvs in clojure? Is it worthwhile to force evaluation of the rows and store in a binding of some kind to access later? Or is it better/more ergonomic to simply read/access the file each time and you can lazily evaluate?

paul.legato19:12:24

It depends a lot on what you’re trying to accomplish in a broader sense. Is it a relatively small CSV that you’ll have to use many times? May as well read it all into memory. Is it a huge CSV that’s too big to read into memory? Maybe process lazily.

Michael Stokley19:12:59

well, i'm not sure how i got here, but i'm starting to feel squeamish about returning functions. because they're not data. i can't "see" what they are - eg #function[workflow.server/eval51723/fn--51726]

Michael Stokley19:12:25

is that the right one? lol idk

paul.legato19:12:05

I always thought it would be cool if functions printed their source code rather than that sort of memory reference

Michael Stokley19:12:16

i guess i could return a quoted expression

paul.legato19:12:18

They’re still data. It’s just not rendered in an easily readable way

paul.legato19:12:42

Down that path lies madness. Then you have to keep track of quoting and unquoting and evaluating and skipping evaluation

🙏 3
paul.legato19:12:28

Also, macros: just functions that run at read-time rather than run-time

seancorfield19:12:35

You can put a name between fn and the argument list, which will make that output a little clearer.

seancorfield19:12:09

user=> (defn foo [n] (fn [x] (* n x)))
#'user/foo
user=> (foo 2)
#object[user$foo$fn__20417 0xb666366 "user$foo$fn__20417@b666366"]
user=> (defn foo [n] (fn times-n [x] (* n x)))
#'user/foo
user=> (foo 2)
#object[user$foo$times_n__20422 0x3113595e "user$foo$times_n__20422@3113595e"]
user=> 

Michael Stokley19:12:34

yeah, that's better

paul.legato19:12:10

If you like the idea of directly manipulating functions-as-data, quoted expressions and trees of symbols, that’s macros.

Michael Stokley20:12:18

thank you for pointing that out

Michael Stokley20:12:13

i'm pretty new to macros, and the macro club rule #1 tends to dampen my enthusiasm

Michael Stokley20:12:22

so it wouldn't have occurred to me.

paul.legato20:12:42

yeah, it’s very cool in itself, but practical applications are somewhat limited. Most things can be done more cleanly using mostly functions, with perhaps a small macro for syntactic sugar.

paul.legato20:12:49

The theoretical scope is astounding, though, isn’t it: your macro can take some arbitrary symbolic code as input and rewrite it in any arbitrary way it likes.

paul.legato20:12:00

The main difference is that functions can’t control whether and how their arguments are evaluated. They don’t get access to the raw symbols, just to the results of evaluating those symbols. Macros get full control at the raw symbol level. This is sadly much less broadly useful than it may seem, though occasionally there are indeed real world use cases where nothing else will do.

paul.legato20:12:54

Toy example for illustration: you can’t write if-then and similar control constructs directly as a function, only as a macro. As a function, (new-if-fn foo (bar) (baz)) will invoke both (bar) and (baz) before the function even starts. You’d have to do (new-if-fn foo (fn [] (bar)) (fn [] (baz))) to delay evaluation. The macro version (new-if-macro foo (bar) (baz)) just gets the symbols, though, so it can choose to not evaluate the losing branch. The same net effect is achievable both ways, but the syntactic sugar is much nicer with the macro.

Michael Stokley21:12:07

thank you for sharing this, it's quite interesting.

👍 3
Michael Stokley21:12:28

someday i'll have the inspiration to really dig into macros.

Michael Stokley21:12:44

> Down that path (syntax quoting) lies madness. Then you have to keep track of quoting and unquoting and evaluating and skipping evaluation it is tricky - i have to deal with this when programmatically building datomic expressions. do you have any thoughts on why datomic would make such extensive use of syntax quoting? would macros fill the same role?

paul.legato22:12:10

I’ve never used Datomic beyond a quick tryout, so I’m afraid I don’t know much about why it does anything

Schmoho19:12:09

sorry to keep bothering, but I now am using an alias

{:path     ["src"],
 :deps
 {org.clojure/clojure #:mvn{:version "1.10.1"}}
 :aliases
 {:bonkers
  {:jvm-opts []}}}
and I'm trying to have the jvm use this option "-Dadd-opens=java.base/jdk.internal.ref=ALL-UNNAMED" but this apparently has no effect - nothing visible in the Properties and the IllegalAccess that causes this shenanigans is still illegal - runtime version 14 here, if that's relevant

seancorfield19:12:55

@d.eltzner How are you invoking the CLI for that? (and presumably you mean :jvm-opts ["-Dadd-opens=java.base/jdk.internal.ref=ALL-UNNAMED"])

Schmoho19:12:49

/usr/local/bin/clojure -Sdeps '{:deps {nrepl {:mvn/version "0.8.3"} refactor-nrepl {:mvn/version "2.5.0"} cider/cider-nrepl {:mvn/version "0.25.5"}}}' -m nrepl.cmdline --middleware '["refactor-nrepl.middleware/wrap-refactor","cider.nrepl/cider-middleware"]' -A:bonkers

seancorfield19:12:01

@d.eltzner -A:bonkers needs to be before the "main opts", i.e., before -m nrepl.cmdline etc.

seancorfield19:12:19

Otherwise it will be read as a command-line argument to nrepl.cmdline itself.

Schmoho19:12:45

😄 yup, I just figured that out. thanks!

Alex Miller (Clojure team)19:12:39

that's not the right syntax for add-opens

Alex Miller (Clojure team)19:12:55

--add-opens is a jvm flag (not a jvm system property)

Alex Miller (Clojure team)19:12:20

so you'd want something like :jvm-opts ["--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED"]

Alex Miller (Clojure team)19:12:01

sorry if I misled you above, I thought you were actually talking about Java system properties

Alex Miller (Clojure team)19:12:32

you won't see that in Java system properties - it's used by the jvm itself as it checks module visibility

Schmoho19:12:52

I see - still helped a lot in that it helped me along with the jvm-opts so can I see these at runtime? because apparently this still didn't do what I hoped it would

Alex Miller (Clojure team)19:12:08

there may be some way via reflection or the module api to introspect those flags, but I don't know

seancorfield19:12:06

See the thread above -- @d.eltzner had the -A:bonkers flag after the main opts for nREPL.

seancorfield19:12:36

(I'm assuming a relatively recent CLI version BTW @d.eltzner -- what does clojure -Sdescribe show?)

Schmoho19:12:52

Some rather interesting information - what would :force and :repro do? and how does the /usr/local deps.edn end up in the config-files? via the :install-dir?

{:version "1.10.1.716"
 :config-files ["/usr/local/lib/clojure/deps.edn" "/home/user/.clojure/deps.edn" "deps.edn" ]
 :config-user "/home/user/.clojure/deps.edn"
 :config-project "deps.edn"
 :install-dir "/usr/local/lib/clojure"
 :config-dir "/home/user/.clojure"
 :cache-dir ".cpcache"
 :force false
 :repro false
 :main-aliases ""
 :repl-aliases ""}

Alex Miller (Clojure team)20:12:59

-Sforce forces classpath cache recomputation

Alex Miller (Clojure team)20:12:19

-Srepro omits the ~/.clojure/deps.edn from the deps.edns that are used

Alex Miller (Clojure team)20:12:37

these are both doc'ed in clj -h and man clj btw

Alex Miller (Clojure team)20:12:04

the :config-files thing is actually a bit of a lie now

Alex Miller (Clojure team)20:12:41

the first location there is created during installation and is now known as the "root" deps.edn in the docs. in reality it's actually read as a resource from the tools.deps library by clj. the file in that location should be the same file, but it is not actually read from the filesystem anymore (but that is there as some older tools, notably datomic ion dev tools) are still loading it from there

Alex Miller (Clojure team)20:12:42

if you have other tools-deps / clj questions, #tools-deps is a better channel for that

Schmoho20:12:17

okay I'll keep it in mind! I'm just trying to pick it up as I go at the moment. by the way what I was up to worked out fine eventually, thank you both very much!

🎉 3
st3fan21:12:26

Does anyone know a library that can (ascii) tabulate/format data so that i can print pretty reports in the repl or cli? Something like https://pypi.org/project/tabulate/

noisesmith21:12:45

there's clojure.pprint/print-table for sequences of maps with the same key

3
st3fan21:12:10

oh! cool 🙂

noisesmith21:12:58

some limitations, but it works for quick and dirty

user=> (print-table [{:a 0 :b 1} {:a 33 :b 12} {:a "OK" :b "bye" :c :hmm}])

| :a |  :b |
|----+-----|
|  0 |   1 |
| 33 |  12 |
| OK | bye |
nil

noisesmith21:12:13

notice it doesn't attempt to show that :c key

noisesmith21:12:56

and of course, for other cases, you can probably use group-by / mapcat etc. to make something print-table can consume

st3fan21:12:25

that is great - exactly what i need

nate21:12:03

print table is amazing, I use it all the time. you can also pass in keys to use to fix the "missing :c" problem:

(print-table [:a :b :c] [{:a 0 :b 1} {:a 33 :b 12} {:a "OK" :b "bye" :c :hmm}])
result:
| :a |  :b |   :c |
|----+-----+------|
|  0 |   1 |      |
| 33 |  12 |      |
| OK | bye | :hmm |

3
noisesmith22:12:44

@st3fan another more robust option is using clojure.data.csv to write a csv output, then use a spreadsheet to view the csv

noisesmith22:12:12

oh, you specifically wanted CLI - so yeah I think you want print-table

paul.legato22:12:35

Wow, I had no idea print-table was a thing. I’m going to use this all the time!

seancorfield22:12:17

I love print-table I use it all the time when I'm REPL'd into production and querying the database 🙂

paul.legato22:12:57

Are there any other somewhat obscure but highly useful functions like that lurking about?

nate22:12:15

there are several

nate22:12:27

frequencies blew my mind when I found out about it

👍 3
nate22:12:40

juxt is a really fun one for making functions with functions

❤️ 3
nate22:12:47

if you'd like to hear more about juxt, check out https://clojuredesign.club/episode/076-multiple-views-on-juxt/ (an episode on my podcast)

dpsutton22:12:55

clojure.set/index is amazing. i see hiredman talk about this one a lot and i think its incredibly useful and underused

seancorfield22:12:19

Maybe follow https://twitter.com/rcfotd on Twitter? Random Clojure Function Of The Day

parrot 3
borkdude22:12:00

Another fun way to discover new core fns: https://borkdude.github.io/re-find.web/

❤️ 6
noisesmith22:12:59

user=> (->> (all-ns) (mapcat (comp vals ns-publics)) rand-nth meta ((juxt :name :doc)))
[load-file "Sequentially read and evaluate the set of forms contained in the file."]

noisesmith22:12:39

realistically you probably want the namespace it came from as well...

borkdude22:12:28

$ bb -e "(->> (all-ns) (mapcat (comp vals ns-publics)) rand-nth meta ((juxt (comp ns-name :ns) :name :doc)))"
[clojure.set union "Return a set that is the union of the input sets"]

💯 9
walterl22:12:39

"Fortune, Clojure edition"