Fork me on GitHub
#clojure
<
2022-01-28
>
bastilla00:01:01

Hi, I posted in #Heroku but I think, that channel is not very busy. So pls bear with me I re-post here for it's really urgent. Hi there. After a fight, I deployed my first app (which compiled remotely) and restore my local postgresql to Heroku. Then issued

heroku ps:scale web=1
heroku open
which fails with logging:
2022-01-27T23:41:43.611429+00:00 app[web.1]: bash: java: command not found
2022-01-27T23:41:43.815109+00:00 heroku[web.1]: State changed from starting to crashed
2022-01-27T23:41:13.127393+00:00 app[api]: Scaled to web@0:Free by user <me>
2022-01-27T23:41:33.763748+00:00 app[api]: Scaled to web@1:Free by user <me>
2022-01-27T23:43:37.962184+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host= request_id=991cd694-e802-4cbc-a374-9f97f5f353c8 fwd="88.130.152.82" dyno= connect= service= status=503 bytes= protocol=https
2022-01-27T23:43:51.379794+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host= request_id=faedadd1-010e-4fef-88da-5e8bba371548 fwd="88.130.152.82" dyno= connect= service= status=503 bytes= protocol=https
First day of Heroku. Somewhat lost here. Also did heroku restart. Has anyone ever experienced this? Thanks for helping out a noob here.

jmckitrick00:01:41

What’s in your Procfile?

bastilla00:01:24

lol, was just about to post it. This is the sole content of it:

web: java -Dclojure.main.report=stderr -cp target/uberjar/my-prj.jar clojure.main -m my-prj.core

bastilla00:01:08

Could it be, I need to activate the Java buildpack?

jmckitrick00:01:03

Does your project file have a build process for the uberjar? Have you tested that part locally?

bastilla00:01:48

It has and I have. It fails, but with something else (not bash: java: command not found ). After lein uberjar this java $JVM_OPTS -jar /home/<BLA>/my-prj/target/uberjar/my-prj.jar gets me:

Exception in thread "main" java.lang.ExceptionInInitializerError
...
Caused by: java.lang.Exception: conman could not find the query file:my-prj/db/sql/queries.sql
I figured this may be irrelevant because dev mode works and it compiles on Heroku.

bastilla00:01:51

{:uberjar {:omit-source true
             :prep-tasks ["compile" ["run" "-m" "shadow.cljs.devtools.cli" "release" "app"]]
             :aot :all
             :uberjar-name "my-prj.jar"
             :source-paths ["env/prod/clj"  "env/prod/cljs" ]
             :resource-paths ["env/prod/resources"]}

jmckitrick00:01:35

I can’t imagine why the java command would not be found, assuming you have the correct buildpack.

bastilla00:01:56

I added Node.js but not anything Java. Just found java/java and added it. About to get results. Btw, THANKS for chiming in. Pressure is intense here.

jmckitrick00:01:11

I’ve been deploying clojure on heroku for a long time, but your issue isn’t one I’ve seen. Sorry I can’t be of more help. I’ll keep this channel open in case you come up with anything else that might point us in the right direction.

bastilla00:01:31

Thanks, Appreciate it. It's a Luminus project, with Clj, Cljs, HugSQL, PostgresQL.

hiredman00:01:24

it sounds like you have a file, querires.sql that contains a bunch of sql for conman, and you you are not correctly packaging it

hiredman00:01:12

my wild guess is it is in ./resources and your :resource-paths in the :uberjar profile is causing that path not to be included in the uberjar

hiredman00:01:23

but I forget how lein does profile merging

bastilla00:01:05

Ok, neither heroku/java nor java/java buildpackages work. Compile fails. Now it's just that lone Node.js buildpackage that's active. @hiredman Yeah thats a local issue. Might be the remote culprit as well. I look into it now.

hiredman00:01:00

if that is happening locally it will be happening remotely as well

jmckitrick01:01:22

@bastilla did you create a new queries.sql or use an existing one? That could explain how the file ended up in the wrong location.

bastilla01:01:59

I placed it where it can be found now. That is: env/prod/resources/myprj/db/sql/query.sql . So the error is gone. But starting up the uberjar gets: 2610 ERROR myprj.core - Database configuration not found, :database-url environment variable must be set before running Which I think is okay, because there is no prod db conf for the db here. (Sidenote, I replace the actual project name here with my-prj, well, myprj, respectively. Sidenote 2, Thanks for helping me out with some brain. It's 2 am here in Germany I my brain is toast!) Now the server compile fails with:

remote: -----> App not compatible with buildpack: 
remote:        bash: /tmp/codon/tmp/buildpacks/006982d415fcc94f95c9a1471c9cbb75a24be96a/bin/detect: No such file or directory
But there is just this Node.js buildpack left.

bastilla01:01:49

Did a

heroku buildpacks:clear
and then added heroku buildpacks:add heroku/nodejs again. It compiles successfully again. But
> heroku ps:scale web=1
Scaling dynos... done, now running web at 1:Free
> heroku open
fails again. Log says:
2022-01-28T01:29:37.971210+00:00 heroku[web.1]: Starting process with command `java -Dclojure.main.report=stderr -cp target/uberjar/myprj.jar clojure.main -m myprj.core`
2022-01-28T01:29:39.204977+00:00 app[web.1]: bash: java: command not found
2022-01-28T01:29:39.354776+00:00 heroku[web.1]: Process exited with status 127
2022-01-28T01:29:39.457410+00:00 heroku[web.1]: State changed from starting to crashed
2022-01-28T01:29:39.465245+00:00 heroku[web.1]: State changed from crashed to starting
2022-01-28T01:29:43.457329+00:00 heroku[web.1]: Starting process with command `java -Dclojure.main.report=stderr -cp target/uberjar/myprj.jar clojure.main -m myprj.core`
2022-01-28T01:29:44.525385+00:00 app[web.1]: bash: java: command not found
2022-01-28T01:29:44.662960+00:00 heroku[web.1]: Process exited with status 127
2022-01-28T01:29:44.716811+00:00 heroku[web.1]: State changed from starting to crashed
2022-01-28T01:29:48.290897+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host= request_id=0d3f6c20-e2d6-4039-8a83-f3ca5b94683f fwd="88.130.152.82" dyno= connect= service= status=503 bytes= protocol=https
2022-01-28T01:29:48.912199+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/favicon.ico" host= request_id=612ad6f1-12cf-42bb-9485-e4d6f1c6775f fwd="88.130.152.82" dyno= connect= service= status=503 bytes= protocol=https             
So I am back to square one.

1
bastilla08:01:43

Ok, found it. Contrary to Herokus doc the buildpack for Clojure (heroku/clojure) is NOT automatically used/active when initiating... well... a CLOJURE project. lol sigh , I had to do this manually (did it via dashboard). Now java is found in a JVM (clj) project. sigh Wow....

p-himik09:01:33

Either your project is deps.edn-based and the buildpack is expecting project.clj, or your build step that builds the uberjar simply removes everything that's not relevant for running the project, including all the project definition files. BTW on Heroku, it's not really necessary to build an uberjar. You can just run your whole project from sources. Some people mention that there might be some problems with that if some dependency for example changes without changing the version or something uses version ranges. Personally, I've never had any problems with sources though, but I definitely had some problems with uberjars.

bastilla09:01:58

uh... mind-blown It's crazy what's possible these days. A thousand ways to do thing, e.g. deploy, configure, run, etc. I just wonder if this makes everyday easier. philosophy-mode off

seancorfield01:01:12

Maybe this could go to the #heroku channel? Or into a thread, at least?

Nom Nom Mousse10:01:06

Is calling a function with side effects from a watcher bad practice? I have a watcher that watches an atom of values. The watcher calls a function that might update the atom with more values. As long as values are added the watcher calls the function again.

p-himik10:01:15

Conceptually, having side effects there is alright because otherwise watcher functions would be nearly useless. But changing an atom from its own watcher is definitely yucky.

p-himik10:01:42

Hide all your atom operations behind some functional interface and always do the necessary recursive mutation that you need to do in a single swap!.

Nom Nom Mousse10:01:36

Thanks! I agree.

Nom Nom Mousse11:01:12

I have the code below. When I post data to /json like this (curl/post full-url {:body "from clojure"}) The request :body looks like this:

#object[org.httpkit.BytesInputStream 0x468aad77 "BytesInputStream[len=12]"]
Am I missing some headers that say this is JSON or what? I was expecting (:body request) to return a map due to the wrap-format middleware.

Nom Nom Mousse12:01:13

I've tried to add headers to the post:

(curl/post full-url {:body "from clojure" :headers {"Content-Type" "application/edn"
                                                        "Accept" "application/edn"}})))
Still it gives the same result.

Nom Nom Mousse12:01:57

I had to look up :body-params not :body. Also, I had to post the JSON as a string:

(curl/post full-url {:body "{:from 'clojure'}"
                         :headers {"Content-Type" "application/edn"
                                   "Accept" "application/edn"}})))

hanDerPeder12:01:39

I assume you want your clojure data sent as json. try this

{:content-type :json
 :form-params {:from "clojure"}
 :as :json}

hanDerPeder12:01:34

:form-params is the data you want to send; your form. :content-type says to encode your data as json. which means passing the data in the body as a json encoded string :as :json say you’re expecting json content to be returned, which will be translated to clojure data when received.

hanDerPeder12:01:15

this is using clj-http, I see you’re using http.kit. haven’t used it but I assume this will work there as well

Nom Nom Mousse12:01:30

I will try. I am actually using babashka for the POSTing, but it seems they use http-kit.

Mark Wardle13:01:17

Hello all. I am sure I am missing a simple solution. I’m passing a java HashMap from a java client into my clojure library and would like to turn it into something that implements clojure.lang.Associative. I could do this clojure-side using (into {} xxx) but I don’t want to pollute my clojure library with that. Is there an easy and forwards-compatible way of invoking that using java? The constructor for clojure.lang.PersistentHashMap looks both complicated and probably only for internal use.

magnars13:01:36

You could look into implementing the clojure.lang.Seqable , clojure.lang.Associative and clojure.lang.ILookup interfaces. edit I see you've already thought about that. Sorry!

👍 2
Alex Miller (Clojure team)14:01:10

You could use the Clojure Java API to call into

Mark Wardle16:01:39

Thanks both - I have used the clojureFn.invoke() with my own functions - but was unsure how to do that with into and an empty clojure persistent map with the constructor from java. But I guess I could do intoFn.invoke(Clojure.read(“{}“), someOtherMap)? Thank you.

👍 1
Mark Wardle16:01:55

I was bitten by an unexpected bug because the java maps were handled beautifully until some niche condition occurred, and my clojure code assoc’ed a replacement value to deal with that niche condition, and it blew up. Until then, I thought I’d be fine passing in immutable associative structures just like I don’t worry about java collections that are sequences, but I’ll carefully marshal maps into clojure maps in my API wrapper to avoid this. Thank you.

manutter5114:01:31

I’ve been writing clojure code since version 1.2, and I still get a kick out of the way the code seems to just naturally fall into simple, succinct, elegant implementations. 😄

🎯 8
manutter5114:01:55

Sorry, I spent years writing PHP (and later, Java), and I just have to get that off my chest every so often.

😉 2
😃 1
😆 2
West15:01:07

What is the current state of Clojure on Android?

JohnJ16:01:26

Via clojurescript and react native

Joshua Suskalo16:01:25

JVM Clojure on android is currently stagnant. The old port still mostly works, and you could probably apply the changes they've made to a modern clojure version (I think they're still on clojure 1.7), but the slow startup time means that JVM clojure on android is less than a good user experience.

vlaaad18:01:38

I heard cljdart is going to be released Q1 2022...

baptiste-from-paris18:01:52

Indeed, we are working hard for this release

👍 2
phronmophobic19:01:01

It should be possible to compile clojure for android using graalvm's native-image. I don't think there are any technical limitations. It's just a matter of setting up a project with the right config. If anyone is interested, you can join #graalvm-mobile or follow this thread, https://clojurians.slack.com/archives/C0260KHN0Q0/p1638032063006400

West20:01:18

I see, yeah I was specifically talking about JVM Clojure on Android. @U5NCUG8NR I figured it was dead. Last time I couldn't get lein droid working. At least there are other ways.

West20:01:21

Now what about Krell. Is that still a thing?

Joshua Suskalo20:01:42

last commit november last year

Joshua Suskalo19:01:47

So while I don't recommend anyone uses this, in response to a conversation with someone I made a macro that can be used to help beginners get past reloading woes before they learn to write reloadable code while working with a web server or other long-running application. https://gist.github.com/IGJoshua/0388076fcc9253575ea7b20f4522fa4d

didibus19:01:52

What does that add on top of the existing Var indirection?

didibus19:01:10

Like old references already point to new one with standard def, so this is for which scenario?

Joshua Suskalo19:01:15

What it gives you is beginners' "expected behavior" when you do something like

(run-jetty app {:port 8080})
where app is a defreloadable handler

Joshua Suskalo19:01:32

Yes, for someone who understands clojure's evaluation model this is solved easily by just throwing a var quote on there

Joshua Suskalo19:01:50

But the purpose of this macro is for people who don't want to get into the weeds about evaluation model to be able to play around with the language without having a bad experience reloading code.

didibus19:01:43

The issue in my mind is actually with run-jetty itself, it shouldn't capture the function, but instead capture the Var, and then normal Clojure re-evaluation would work fine. But I see where you're coming from.

Joshua Suskalo19:01:03

well then that'd require run-jetty to be a macro

didibus19:01:16

I'm just not sure if its easier to tell a beginner to use this macro for all defns, instead of just telling them to do (run-jetty '#handler ...)

Joshua Suskalo19:01:31

Well what prompted me to write this macro in the first place was someone having reloading issues when they already had done the var quote on run-jetty. It was the routing library that was causing them issues, because the functions the router dispatched to were in a different file from the handler definition, so when they reloaded the implementation functions, the router didn't pick the changes up because they didn't re-evaluate the handler.

Joshua Suskalo19:01:37

This would have solved that problem

didibus19:01:18

Run jetty doesn't need to be a macro.

user=> (defn foo
  #_=>   [bar]
  #_=>   (bar))
#'user/foo
user=> (defn bar
  #_=>   []
  #_=>   1)
#'user/bar
user=> (foo bar)
1
user=> (defn bar
  #_=>   []
  #_=>   2)
#'user/bar
user=> (foo bar)
2

didibus19:01:59

Oh, nevermind, ya I see what you mean

👍 1
didibus19:01:19

I wonder how hacky this would be for run-jetty and others to do internally:

(-> handler .getClass .getSimpleName (str/split #"\$") (->> (str/join #"/")) symbol requiring-resolve)

Joshua Suskalo19:01:04

because that wouldn't support anonymous functions

didibus22:01:32

Well you can detect those too and fallback to capturing the function in those cases. But ya, I also do think it would be pretty hacky, but only if Clojure doesn't make the class name convention something to rely on.

didibus22:01:00

But also probably not super portable, so 😝

jumar05:01:08

What I would (still) appreciate is a concise description/ guide describing these issues and when the var indirection is used and when the actual value is captured so I need to be careful Is there some official resource for that?

Joshua Suskalo14:01:24

@U06BE1L6T when you are making like a data literal or something and you put a function name in it and the data structure is evaluated (and since it's a data structure it evaluates to itself), that means that the function name is looked up as a var, and because it's a var it gets dereferenced, and what's in the var is a function object. So now you have a data structure with a function object in it. If you now go and change the function and re-evaluate it, a brand new function object is created and put into the var. The data structure isn't updated because it doesn't know about the var, it just has a function object that points to what the programmer now thinks of as outdated code. This is the root cause of all the var indirection reloadability woes. Any time you pass a function as an argument to the construction of a persistent object, or to the start of a long-lived process, you'll run into this "problem". In these cases the way to fix the problem is to introduce a level of indirection, so that whenever the code wants to call the function, it has to first look up the var. Well as it turns out vars implement the IFn interface, and what they do when called as a function is dereference themselves and then call the result as a function. So in most of these cases you can get away with just passing a function with a var quote, and now you're not passing a function, you're passing a var. The reason this sometimes gets confusing is because you can have one snippet of code where different lines are executed at different times, either because it's in a macro, or because some code is in a lambda and some is not. Vars are looked up when the code is run, not when it's compiled (besides to just validate that you're not referencing an unbound var), so that means that you can have code in a lambda that "reloads just fine" while other code does not, and that's all because of when the evaluation happens.

Joshua Suskalo14:01:38

This is why in the macro I say the solution is to learn the Clojure evaluation model.

jumar07:02:45

@U5NCUG8NR I forgot to say thank you 🙂 I'm quite well aware of the issues since I've been working with Clojure for a few years and ran into both "jetty handler" issue and "function captured in a data structure) issue multiple times. I also sort of knew how this is related to the clojure evaluation model but your post helped to clarify these things definitely You should consider writing a short article on this topic - I think it would be very valuable.

Joshua Suskalo14:02:17

I've been looking for content I should put on a blog for a while to get motivation to make one, and I think that's it then. Thanks!

didibus20:02:23

@U06BE1L6T If you've never read through the Clojure official reference, I highly recommend doing so. This section covers Vars: https://clojure.org/reference/vars

didibus20:02:28

You also need to read the evaluation reference to understand the full picture: https://clojure.org/reference/evaluation

didibus20:02:24

It's pretty simple once you know it. When symbols are resolved: > A lookup is done in the current namespace to see if there is a mapping from the symbol to a var. If so, the value is the value of the binding of the var referred-to by the symbol. See how a symbol mapped to a Var doesn't return the Var, but what the Var is bound too. That means if its mapped to a Var bound to a function, you get the function instance back, not the Var, thus the Var indirection is lost every time a symbol is resolved. To keep the indirection, you need to explicitly Var-quote the symbol, then when it gets resolved it will return the Var, keeping the indirection.

didibus20:02:20

When you re-def, you are not changing the Var mapped to the symbol, but the binding of the Var that already maps to that symbol. This is how old reference to the symbol automatically see the change, because when they get resolved again, they will retrieve the same Var, but it will now point to the new function instance. Where you get in trouble is if you are not keeping track of the symbol or the Var, if instead you captured the function instance, then you just have a direct reference to the old function, and nowhere to lookup for the new one. If instead you captured the symbol or the Var, then you'd be able to lookup the new function.

Joshua Suskalo19:01:37

I really don't like it because it obscures Clojure's evaluation model, but it could be useful for beginners just getting their feet wet with http-kit and reitit or other routing libraries that rely on data with function references in them.

ghadi19:01:07

bandaids vs. problem removal

ghadi19:01:54

what interferes with writing webservers that are responsive to REPL eval?

Joshua Suskalo19:01:18

A lack of understanding of the evaluation model

Joshua Suskalo19:01:38

the point of the macro is just for people who don't understand that model and for whom it is too early to introduce it

ghadi19:01:24

I hear you, but I'm not sure that's the only factor

seancorfield20:01:13

Personally, I think telling them to use #' for function names passed into stuff like that is much better than introducing a non-standard macro in a library. Especially as the #' stuff is explained in the official docs that teach folks about Clojure and the REPL: https://clojure.org/guides/repl/enhancing_your_repl_workflow#writing-repl-friendly-programs

Joshua Suskalo20:01:09

oh yeah, I agree, I just made it as a response to someone asking if it were possible to avoid needing to keep track of the difference between passing functions and passing vars after they had some issues with reloading their handler. It's why I specifically put in the docstring not to use it and just to learn the evaluation model.

seancorfield20:01:38

(defmacro cool-stuff "Do not use this macro. Learn stuff properly instead!" [...] ...) :rolling_on_the_floor_laughing:

😂 1
dpsutton19:01:04

And would finding docs for that be easier than understanding vars?

Joshua Suskalo19:01:26

I mean probably not because this is a tiny gist I'm putting up somewhere, but more or less I'm putting something out to gauge interest in it, because maybe it could be something useful for someone writing a webserver tutorial or teaching a clojure workshop.

pinkfrog23:01:29

I don’t know if it is technically hard to implement preemptive go routines or it is designed to do so? If the latter, what’s the benefit?

phronmophobic00:01:13

I think this would be technically challenging and probably a lot of work. Some ideas for how this might be implemented: • Write your own clojure compiler that inserts checkpoints for all function calls and looping constructs • Write a clojure compiler that targets a runtime that already supports preemption • Write your own clojure interpreter • Wait for project https://openjdk.java.net/projects/loom/ to ship and hope they do all the hard work for you.

thanks3 1
phronmophobic00:01:46

There are also partial solutions where you write a macro that automatically adds interruption checks at function calls and loops, but may fail to be interruptible if other code the macro can't rewrite is called

hiredman23:01:46

the way to do wit would be to yield the thread executing the go routine more often then just on channel operations

hiredman23:01:28

which is doable, but will perform worse

p-himik23:01:36

For my own education - what does "preemptive go routine" mean?

phronmophobic23:01:48

wouldn't you also need to prevent the go routine from calling any function that is also not provably preemptible?

hiredman23:01:27

ah, right, so that wouldn't work

hiredman23:01:20

preemption would be like how threads work

hiredman23:01:34

in that you have many threads running on on a cpu, and you don't wait for a thread to yield control of the cpu of its own accord to schedule another thread to run