Fork me on GitHub
#clojure
<
2022-02-14
>
Faris03:02:42

Hi, a bit of a long shot here asking this, but maybe someone else might have encountered it before. My project won’t run after I upgraded to a M1 Macbook. Seems like the failure is on the postal library. https://github.com/drewr/postal

Syntax error compiling . at (postal/message.clj:41:14).
Syntax error (ClassNotFoundException) compiling . at (postal/message.clj:41:14).
javax.activation.DataHandler
Looking at the error message, I found an issue that matches the error perfectly. https://github.com/drewr/postal/issues/107 However, applying the workaround seems to break to break other things in my app, I keep getting the error below
Cannot invoke "java.util.concurrent.Future.get()" because "fut" is null
The weird thing is I am using JDK 8 but the issue above says it is a JDK9+ issue. I am pretty stuck on how to proceed further so any tips would be greatly appreciated. Thanks

simongray08:02:41

I’m unsure why you would experience this issue on 8 (maybe one of your dependencies is causing it), but… why don’t you just upgrade the JVM? Version 17 is faster and is also an LTS version. Any JVM version >=9 will have that issue as it’s related to several classes being taken out of the Java core libraries and put in separate modules. Come join us in the future 😄 I’m also on an M1 Macbook.

👍 1
simongray08:02:25

Oh sorry, I didn’t see that you already updated the library and are still experiencing an issue. I guess you need to take your issue to somewhere more postal-specific, like a Github issue?

💯 1
Faris08:02:11

Thanks for the reply, yeah upgrading is definitely the way I would go. Just need to get buy-in from the rest of the tech team.

lvh03:02:34

I’m having some difficulty creating a custom SQLite function in Clojure; I think the issue is proxy can’t call protected superclasses; I’d really like to avoid gen-class. I think this means I have to use reflection to call a Method directly? code in 🧵

lvh03:02:38

(ns com.latacora.steak-knives.sqlite
  (:require    [next.jdbc :as jdbc])
  (:import (org.sqlite Function)))

(def my-fn
  (proxy [Function] []
    (xFunc []
      (println "yo")
      (let [;;argc (.args this)
            x (.value_double this 0)
            y (.value_long this 1)
            res (+ x y)]
        (printf "xFunc called with %s args: %s and %s, result %s" 0 x y res)
        (.result this res)))))

(def conn
  (-> {:dbtype "sqlite" :dbname ":memory:"}
      (jdbc/with-options  jdbc/snake-kebab-opts)
      jdbc/get-connection))

(Function/create conn "myFunc" my-fn)

(jdbc/execute! conn ["SELECT myFunc()"])

;; 1. Unhandled org.sqlite.SQLiteException
;;    [SQLITE_ERROR] SQL error or missing database
;;    (java.lang.IllegalArgumentException: No matching method result found taking 1
;;    args for class
;;    com.latacora.steak_knives.sqlite.proxy$org.sqlite.Function$ff19274a)

lvh03:02:20

I know proxy warns: > Note that while method fns can be provided to override protected methods, they have no other access to protected members, nor to super, as these capabilities cannot be proxied. but this seems like an in-between case because it’s a method (but I’m calling not overriding)

lvh03:02:32

I’ll try to implement them explicitly and use proxy-super and see what happens

lvh03:02:20

calling proxy-super in xFunc (on result etc) doesn’t work; I just get an error suggesting it couldn’t find such a method even though it deifnitely exists

lvh03:02:34

I assume that error is due to visibility

hiredman04:02:37

Try (.result (identity this) res)

hiredman04:02:27

ugh, I mean, I doubt that will work

lvh04:02:26

nope 😞

lvh04:02:29

(def my-fn
  (proxy [Function] []
    (xFunc []
      (println "yo")
      (let [argc (.args (identity this))
            x (proxy-super value_double 0)
            y (proxy-super value_long 1)
            res (+ x y)]
        (printf "xFunc called with %s args: %s and %s, result %s" argc x y res)
        (proxy-super result res)))))

lvh04:02:31

results in:

lvh04:02:46

[SQLITE_ERROR] SQL error or missing database
   (java.lang.IllegalArgumentException: No matching field found: args for class
   com.latacora.steak_knives.sqlite.proxy$org.sqlite.Function$ff19274a)

lvh04:02:00

I really wish the error was clearer about the lack of matching field

hiredman04:02:31

Well, it is not matching field or method

lvh04:02:35

the native Java way:

Function.create(conn, "myFunc", new Function() {
          protected void xFunc() {
              System.out.println("myFunc called!");
          }
      });

phronmophobic04:02:08

It's usually fairly easy to include a bit of java in your clojure project. Either with lein javac or with https://clojure.org/guides/tools_build#_mixed_java_clojure_build. I've had a few cases where I wrote a small, clojure friendly wrapper around Java boilerplate that would otherwise be very awkward to express in clojure.

bastilla13:02:31

I encountered a Heroku server cache phenomena. (Forgive me for not using #heroku but that channel seems dead.) So, my heroku build uses one old source file, even though it's been updated in my git commit. That source file is not a regular Clojure source but a SQL file used by HugSQL - "queries.sql". This will get transposed into clj code and then complied. Works on my dev machine. Worked the first time around on Heroku!! Now "old" functions I didn't alter since my first project deploy still work, but new ones are claimed to be unbound.

ERROR playgreen.middleware - Attempting to call unbound fn: #'playgreen.db.core/medium-vec
This indicates a kind of server cache issue to me. (Could be wrong, of course.) Has anyone ever dealt with this, or knows how to tell Heroku to please use and also (re-)compile that altered "queries.sql" file for the build process? Note, I stumpled across Herokus clear cache mechanism and applied that - to no avail. (https://help.heroku.com/18PI5RSY/how-do-i-clear-the-build-cache) Greatly appreciate your input! (Pls reply via thread, thanks) (Edit: Opened a ticket at Heroku, but a) they respond slowly, and b) this seems to be so Clojure related that I fear, they'll throw in the towel.)

1
javahippie13:02:48

When you mentioned the git commit, I guess that you use the default Heroku Runner?

bastilla13:02:16

Yes. (I guess.) That is simply pushing it to Heroku via CLI. git push heroku master

javahippie13:02:03

I would be unusual for Heroku to cache anything, as every push should create a new instance under the hood that does not know anything about the others. If you clone from the heroku master branch into a new dir, do the files look as you expect them?

bastilla13:02:31

good idea. Let me check

javahippie13:02:42

Also: Are you running your app in a cluster, is it possible that for some time the ‘old’ version still exists and accepts workloads meant for the new one?

bastilla13:02:53

No cluster mechanism active on this project. It's just a small test site. New code (clj and cljs) does get recognized. It's just this queries.sql file. Just did a git clone from heroku's repos. And that queries.sql file there is up to date. Flubbergasted.

😲 1
bastilla13:02:59

anyway, thanks! for chiming in, @U0N9SJHCH . I'll wait for the support line's response. Guess, I hit a weak spot within Heroku.

javahippie13:02:03

Best of luck with the support! Would be interesting to hear the root cause 🙂

bastilla13:02:51

yeah, sure. I'll post that here!

p-himik14:02:44

@U0N9SJHCH You're incorrect about cache - Heroku caches stuff depending on the buildpack. The way you deploy it matters much less, if at all. That's exactly why the $CACHE_DIR environment variable is available for buildpacks during the build stage. Usually it's used for things like node_modules and .m2. @U01BEC6NUSY Which buildpack are you using for Clojure? Are there any other buildpacks?

💡 1
jcf15:02:48

@U01BEC6NUSY HoneySQL HugSQL generates those functions from SQL when you ask it to. Are you building a JAR during deployment? Just wondering if the file’s there when building your slug but missing at runtime…

jcf15:02:29

Can you start a REPL on Heroku and see if the SQL file is accessible?

jcf15:02:03

And if so, invoke the HugSQL function for defining those DB fns. See if that errors out. https://hugsql.org/using-hugsql/def-db-fns

jcf15:02:04

To automate defining those vars when building the Heroku slug would depend on build tool. If you're using Leiningen you could hook into the uberjar-building process. With Boot and tools.deps it's also doable.

bastilla15:02:49

Sry for my late response. @U2FRKM4TW I have two buildpacks: 1) heroku/nodejs 2) heroku/clojure (that order). Didn't touch any env var, such as $CACHE_DIR. Those buildpacks are pre-defined and I would'nt even know how to alter them. Pls note, this deploy of HugSQL .sql file worked flawlessly the first time(s) I deployed. It's just now, after altering queries.sql that heroku decided to ignore my new SQL stuff. @U06FTAZV3 Heroku builds a uberjar duing deployment, yes. I checked the remote repos and this file in question - "queries.sql" - is up to date. Wether it (or it's compiled source) is also present at runtime also is a good question. Sadly I never understood the whole REPL driven aspect of Clojure. (Read 100x all the what/how/wheres but never got beyond starting a REPL a couple of times and issueing some commands. So I really don't know how to connect to a remote test server, let alone check things once on it. And tbh I don't think this is possible at Heroku.) Here's the deploy output:

$ git push heroku master
Objekte aufzählen: 43, fertig.
Zähle Objekte: 100% (43/43), fertig.
Delta-Kompression verwendet bis zu 4 Threads.
Komprimiere Objekte: 100% (19/19), fertig.
Schreibe Objekte: 100% (23/23), 2.08 KiB | 2.08 MiB/s, fertig.
Gesamt 23 (Delta 13), Wiederverwendet 0 (Delta 0), Pack wiederverwendet 0
remote: Compressing source files... done.
remote: Building source:
remote: 
remote: -----> Building on the Heroku-20 stack
remote: -----> Using buildpacks:
remote:        1. heroku/nodejs
remote:        2. heroku/clojure
remote: -----> Node.js app detected
remote:        
remote: -----> Creating runtime environment
remote:        
remote:        NPM_CONFIG_LOGLEVEL=error
remote:        NODE_VERBOSE=false
remote:        NODE_ENV=production
remote:        NODE_MODULES_CACHE=true
remote:        
remote: -----> Installing binaries
remote:        engines.node (package.json):  unspecified
remote:        engines.npm (package.json):   unspecified (use default)
remote:        
remote:        Resolving node version 16.x...
remote:        Downloading and installing node 16.14.0...
remote:        Using default npm version: 8.3.1
remote:        
remote: -----> Restoring cache
remote:        - node_modules
remote:        
remote: -----> Installing dependencies
remote:        Installing node modules
remote:        
remote:        added 99 packages, and audited 100 packages in 4s
remote:        
remote:        3 packages are looking for funding
remote:          run `npm fund` for details
remote:        
remote:        found 0 vulnerabilities
remote:        
remote: -----> Build
remote:        
remote: -----> Caching build
remote:        - node_modules
remote:        
remote: -----> Pruning devDependencies
remote:        
remote:        up to date, audited 8 packages in 876ms
remote:        
remote:        found 0 vulnerabilities
remote:        
remote: -----> Build succeeded!
remote: -----> Clojure (Leiningen 2) app detected
remote: -----> Installing JDK 1.8... done
remote: -----> Installing rlwrap... 
remote:        Get:1  focal-pgdg InRelease [86.6 kB]
remote:        Hit:2  focal InRelease
remote:        Get:3  focal-security InRelease [114 kB]
remote:        Get:4  focal-pgdg/main amd64 Packages [364 kB]
remote:        Get:5  focal-updates InRelease [114 kB]
remote:        Get:6  focal-updates/main amd64 Packages [1,973 kB]
remote:        Fetched 2,651 kB in 1s (2,477 kB/s)
remote:        Reading package lists...
remote:        Reading package lists...
remote:        Building dependency tree...
remote:        The following NEW packages will be installed:
remote:          rlwrap
remote:        0 upgraded, 1 newly installed, 0 to remove and 21 not upgraded.
remote:        Need to get 0 B/98.2 kB of archives.
remote:        After this operation, 309 kB of additional disk space will be used.
remote:        Download complete and in download only mode
remote: -----> Installing Clojure 1.10.0.411 CLI tools
remote:        Downloading and expanding tar
remote:        Installing libs into /tmp/build_67701169/.heroku/clj/lib/clojure
remote:        Installing clojure and clj into /tmp/build_67701169/.heroku/clj/bin
remote:        Installing man pages into /tmp/build_67701169/.heroku/clj/share/man/man1
remote:        Removing download
remote:        Use clj -h for help.
remote: -----> Using cached Leiningen 2.9.1
remote:        Writing: lein script
remote: -----> Building with Leiningen
remote:        Running: lein uberjar
remote:        Downloading Leiningen to /app/.lein/self-installs/leiningen-2.9.1-standalone.jar now...
remote:          % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
remote:                                         Dload  Upload   Total   Spent    Left  Speed
remote: 100   663  100   663    0     0  44200      0 --:--:-- --:--:-- --:--:-- 44200
remote: 100 13.9M  100 13.9M    0     0   145M      0 --:--:-- --:--:-- --:--:--  145M
remote:        Compiling playgreen.config
remote:        Compiling playgreen.core
remote:        Compiling playgreen.db.core
remote:        Compiling playgreen.env
remote:        Compiling playgreen.handler
remote:        Compiling playgreen.layout
remote:        Compiling playgreen.middleware
remote:        Compiling playgreen.middleware.formats
remote:        Compiling playgreen.nrepl
remote:        Compiling playgreen.oauth
remote:        Compiling playgreen.routes.home
remote:        Compiling playgreen.routes.oauth
remote:        Compiling playgreen.routes.services
remote:        Compiling playgreen.services.read
remote:        Compiling playgreen.validation
remote:        [:app] Compiling ...
remote:        [:app] Build completed. (227 files, 90 compiled, 0 warnings, 42.50s)
remote:        [:adminapp] Compiling ...
remote: :adminapp] Build completed. (230 files, 93 compiled, 0 warnings, 30.90s)
remote:        Created /tmp/build_67701169/target/uberjar/playgreen-0.1.0-SNAPSHOT.jar
remote:        Created /tmp/build_67701169/target/uberjar/playgreen.jar
remote: -----> Discovering process types
remote:        Procfile declares types -> web
remote: 
remote: -----> Compressing...
remote:        Done: 261.4M
remote: -----> Launching...
remote:        Released v17
remote:         deployed to Heroku
remote: 
remote: Verifying deploy... done.

p-himik15:02:16

> don't think this is possible at Heroku It is. Not that long ago, they've added the ability to connect to running dynos - heroku ps:exec.

bastilla15:02:47

Ah, good to know. 🙂 Maybe one day I can wrap my head around this stuff.

p-himik15:02:19

Another guess - maybe your process of building an uberjar also places it inside .m2, "installs" it. In that case, it will be cached and potentially might not be overwritten if the new jar has the same version. Although, that's probably not the case given that the cache purging that you mention didn't work.

bastilla15:02:14

PS to last post: Just found heroku run lein repl.

bastilla15:02:25

Yeah, I purged the cache. .m2 and nodejs stuff got purged, etc.

p-himik15:02:12

heroku run creates a new dyno that has the same files but doesn't let you inspect a running process since there are no CLJ processes - it's basically a copy of the same image but not the same container. Although it might be acceptable in this case. Also, you can just heroku run bash and subsequently do lein repl if you also need some shell commands.

bastilla15:02:13

ah, okay. Just did within Calva (Terminal) and this kind of crashed. Redoing it in a pure shell/bash (Linux) and that is doing what you are describing. Well it ended with

nREPL server started on port 35405 on host 127.0.0.1 - 
So, this isn't exactly remote, I guess?!

p-himik15:02:15

If it doesn't give you a prompt, then probably. I don't really use that functionality, so don't know. In any case, you can use heroku ps:exec with --ssh - that should enable port forwarding which, if set up correctly, will allow you to connect to that REPL from your local machine. Alternatively, you can try to figure out how to run a local REPL with lein.

bastilla16:02:18

Guess I am a huge step forward now. @U2FRKM4TW your first idea worked to get me a remote REPL. (`heroku run bash` and then lein repl). @U06FTAZV3 I did a (hugsql.core/def-db-fns "playgreen/db/sql/queries.sql") and that returned nil instead of Can not read file when giving an incorrect file path. So I guess, it successfully got the HugSQL definitions it found there.

👍 1
bastilla16:02:03

Affirmative. It is there! This

bastilla16:02:33

oops. Again. This

(doc playgreen.db.core/medium-vec)
-------------------------
playgreen.db.core/medium-vec
([] [params] [db params options & command-options])
  Liste aller Medien
proves it.

bastilla16:02:34

This is one of the fns in question that are missing. I must say this kind of error hunting is really entertaining. Learning a lot.

bastilla16:02:03

Regarding @U06FTAZV3 > To automate defining those vars when building the Heroku slug [...] using Leiningen you could hook into the uberjar-building process. I guess there are two possible places where to tell Heroku to please also consider HugSQL text files: project.clj and Heroku's buildpacks. Since "queries.sql" compiled with the initial deploy and than (seemingly) not any more, my wild guess is adjusting the heroku/clojure buildpack might be the right place.

p-himik16:02:05

I don't think so. If HugSQL can modify exsting namespaces then that file should be on your classpath when building the uberjar - then it'll simply be a part of the uberjar and the namespaces will have the right functions without having to write anything to disk. If HugSQL needs some stuff to be run in advance to generate files on disk, then that has to be done before the uberjar is built and those files must end up on the classpath. Read about Heroku Clojure buildpack (maybe its source code if its documentation is unsufficient) and see how you can customize the building process. I can see from its source that there are at least two useful environment variables - BUILD_COMMAND and LEIN_BUILD_TASK.

👆 1
💯 1
bastilla16:02:33

Just processing your input. As a side note: the file "queries.sql" is wihtin src so in classpath.

bastilla19:02:13

Meanwhile I figure it might have to do with AOT compiling. In project.clj there is :main ^:skip-aot playgreen.core. And that source has the line responsible for doing the translation of "queries.sql" to code: (conman/bind-connection *db* "playgreen/db/sql/queries.sql") So I figured maybe that code never gets executed since it's not AOT handled. But placing that snippet to the (conman/connect! ... area didn't yield anyhing. (EDIT: deleted non-constructive line of frustration)

bastilla17:02:21

Ok, got some time now to post the solution. In DEV env, files on src are on classpath and can be evaluated. In TEST/PROD env on Heroku, there is the uberjar in charge that is free of src. So for Heroku such text files as queries.sql must be placed here: /env/prod/resources

1
emccue15:02:34

i feel like i remember some way to get the underlying HttpServletRequest in a ring handler, but i'm missing it now

emccue15:02:57

currently working with ring.adapter.jetty

p-himik15:02:14

Wasn't it attached as metadata?

emccue15:02:47

(defn handler
    [req]

    {:status 200
     :headers {}
     :body (with-out-str
             (clojure.pprint/pprint  [ req
                                      {"METADATA" (meta req)}]))})

emccue15:02:54

[{:ssl-client-cert nil,
  :protocol "HTTP/1.1",
  :remote-addr "[0:0:0:0:0:0:0:1]",
  :headers
  {"sec-fetch-site" "none",
   "host" "localhost:3216",
   "user-agent"
   "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.87 Safari/537.36",
   "sec-fetch-user" "?1",
   "connection" "keep-alive",
   "upgrade-insecure-requests" "1",
   "accept"
   "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
   "accept-language" "en-US,en;q=0.9",
   "sec-fetch-dest" "document",
   "accept-encoding" "gzip, deflate, br",
   "sec-fetch-mode" "navigate",
   "sec-gpc" "1",
   "cache-control" "max-age=0"},
  :server-port 3216,
  :content-length nil,
  :content-type nil,
  :character-encoding nil,
  :uri "/",
  :server-name "localhost",
  :query-string nil,
  :body
  #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x60d6e462 "HttpInputOverHTTP@60d6e462[c=0,q=0,[0]=null,s=STREAM]"],
  :scheme :http,
  :request-method :get}
 {"METADATA" nil}]

emccue15:02:57

does not seem so

😔 1
emccue15:02:44

source has merge-servlet-keys but idk why im not getting them

emccue15:02:52

have to make this work in pedestal too and this is not encouraging

hiredman16:02:43

It isn't part of the ring spec, so you cannot expect it

hiredman17:02:27

ring-jetty-adaptet uses ring-servlet, but doesn't call merge-servler keys

hiredman17:02:21

merge-servlet-keys would only be called if you built your ring app into a war (used defservice to gen class a servlet class, etc)

emccue17:02:58

okay so changing my question - has anyone successfully done an auth0 integration using their java api (and not the rest apis directly)

hiredman17:02:21

the very end

Colin P. Hill20:02:50

(ns foo)
(ns bar)
(require '[foo :as f])
(= ::f/baz :foo/baz)
;; => true
(= 'f/baz 'foo/baz)
;; => false
Is there any way to resolve namespace aliases on quoted symbols so that the above, or something not many more characters than the above, returns true? I'm working on something in the REPL that involves symbol equality, and being unable to type the symbols concisely is causing some friction

dpsutton21:02:29

bar=> (= `f/baz 'foo/baz)
true
bar=> `f/baz
foo/baz

Colin P. Hill21:02:02

ah of course! thank you

Alex Miller (Clojure team)21:02:50

and in 1.11 you'll be able to (require '[foo :as-alias f]) and get same result, but without loading foo