Fork me on GitHub
#clojure
<
2020-04-14
>
Parenoid01:04:59

how do I access a file that is external to a jar file on a server in clojure?

seancorfield01:04:25

@patrickanium and the full file path -- which you may have to build from the current directory (which you can get via Java interop from the environment or system properties... I'd have to look that up).

seancorfield01:04:27

(System/getProperty "user.dir") will tell you where the program was started.

Parenoid01:04:56

cool, thanks!

Parenoid01:04:44

worked like a charm. you are the best.

charch02:04:38

I defrecord'd a Beta and want to try to cast (call?) it using the below syntax after binding it with a let statement. Is this possible?

(let [thing Beta]
  (simulate (->thing {:alpha 1 :beta 3})))
;; => doesn't work because it doesn't know what symbol thing is

rutledgepaulv02:04:21

(let [thing map->Beta] (simulate (thing {:alpha 1 :beta 3}))) should work

charch02:04:56

This is neat. I had to do ->Beta instead. Thank you šŸ™‚

šŸ‘ 4
rutledgepaulv02:04:31

defrecord doesn't produce a function bound to the record's name. it interns ->Beta and map->Beta function vars in the ns the defrecord is in

amalantony07:04:30

Iā€™m serving a file response from a ring handler like below:

(future
      (Thread/sleep (* 5 60 1000)) ; sleep for 5 minutes
      (io/delete-file tmp-file-path))
(resp/file-response tmp-file-path)
I wonder if this will break if my response takes longer than 5 minutes to complete since Iā€™m deleting the tmp-file after 5 minutes. The files can be quite large in size and Iā€™m doing this so that my disk does not fill up with temp files. From looking at ring.util.reponse, calling res/file-response results in a response body that looks like this:
{:status 200
    :headers {}
    :body (io/file tmp-file-path)}
Does io/file (which returns a .File object) make a ā€œcopyā€ of the original file? Since Linux makes a copy of the file on open, Iā€™d like to know if the same behaviour extends to .File objects. Thanks!

phronmophobic07:04:05

pretty sure http://java.io.File is just a description and doesnā€™t make a copy. the file doesnā€™t even need to exist

amalantony07:04:14

@U7RJTCH6J If that is the case, how is the file ā€œstreamedā€ to the client as part of the HTTP response? Does the web-server do this?

amalantony07:04:21

So, then since the file is opened, does deleting the file while the response is still active cause a failure scenario?

phronmophobic07:04:46

good question, it may even be OS dependent

phronmophobic07:04:22

not sure how youā€™re creating the temp file, but you may alternatively pass an inputstream for the body of a response

phronmophobic07:04:50

the inputstream will be closed when the file is finished serving so you can wrap the input stream to delete the temp file when itā€™s done sending the file. unfortunately, Iā€™m not sure how the error cases work in that situation

amalantony07:04:51

@U7RJTCH6J Linux keeps the file handle active even when it is deleted, so Im guessing this should work since my server runs Linux. Iā€™ll look into InputStream (new to the whole Java ecosystem). Thanks for your help!

šŸ‘ 4
hindol07:04:58

I am curious too. When you get around to trying, please post back the results.

hindol07:04:26

As for the JVM, I am guessing the file will be opened via a system call underneath, so Linux's behaviour of retaining the file till it's open kicks in. That's my guess.

šŸ‘ 4
amalantony08:04:00

@UJRDALZA5 Sure, Iā€™ll comment back here once I confirm it. Although, Iā€™m reasonably certain this should work: https://stackoverflow.com/a/2031100/761350

amalantony09:04:12

@UJRDALZA5 @U7RJTCH6J tested this on MacOS. I deleted the file on disk while it was still being served in the HTTP Response. Seems to work fine šŸ‘

šŸ‘ 8
parrot 4
hindol09:04:26

That's great to know!

joshkh08:04:34

can i call a java function on an object using a string as the function name? for example (.player Game "neo") where "player" is def'ed in a collection of available function names.

p-himik08:04:39

You can, but if that string is only available at run time then you will have to use Java reflection.

joshkh08:04:54

how would i go about using reflection to call the function? i'm not worried about performance in this case.

joshkh09:04:45

perfect, thanks a lot for your help

flipmokid10:04:50

Hi, I'm trying to implement an Azure Function App (HTTP trigger) using Clojure. Previously I had a Java class act as the main entry point but ideally, I'd like to have a pure Clojure solution. Azure uses reflection to determine what type to coerce the request body to, eg. the first argument to the method would be something like HttpRequestMessage<Optional<String>> to convert the body from bytes into a string, if present. Is there anyway in Clojure to hint/specify this so that the resulting class has the generic type specified?

hindol10:04:23

I don't know Azure Function so my suggestion can be off, but did you try creating the exact Java type using Java interop?

flipmokid11:04:33

Hi. I created a type using deftype but I'm not sure how to type hint java classes with generics. I can add a type hint of ^HttpRequestMessage before the first function arguemt but not 100% sure how specify the Optional<String> part

hindol11:04:18

Is it a type you receive in your code or you want to create this type to pass to Azure Functions?

hindol11:04:59

If you are receiving this argument in your code, you can type hint it to avoid the need for reflection.

hindol11:04:33

Type hint is a hint for the compiler, it won't change the type of the thing you have to the hinted type.

hindol11:04:27

Regarding Optional<String>, just hint it ^Optional. Java uses type erasure and at runtime when Clojure code will be interacting with the Java code, it is just Optional<Object>.

flipmokid11:04:41

It's a parameter for the function and the azure framework does a bit of reflection to work out the conversions it needs to do. Would something like this work then? ^HttpRequestMessage[Optional[String]] ?

hindol11:04:44

> It's a parameter for the function Sorry, I have zero knowledge of Azure Functions. Is this an API AF provides and you are calling it?

hindol11:04:42

No, you just type hint the outermost thing. So that should be ^HttpRequestMessage.

flipmokid11:04:21

So with Azure functions you create a class and annotate one of your methods as being the method that should be called when the lambda executes. The method you annotate should have the first parameter as HTTPRequestMessage<Optional<T>> and it uses reflection to figure out what T is and tries to convert to that value. So I think the problem I have at the moment is that Azure can't figure out what it needs to convert to as I'm missing some type info.

hindol11:04:31

And when you actually take out the thing inside the request, you type hint that thing with Optional and so on.

hindol12:04:18

Okay, so this is about Java being able to call Clojure code. I understood it wrong earlier.

hindol12:04:59

So, a Clojure function is not a Java function at all. A Clojure function is actually a class implementing IFn which is the invoke()ed.

flipmokid12:04:58

Yeah I have seen that. I was under the impression (probably incorrectly) that using deftype creates the corresponding methods in the class too

hindol12:04:36

What I think (but have never done this) is you need a Java method with the correct signature which will then call your Clojure code.

hindol12:04:10

What you say might be true as deftype creates a Java type. But I have limited knowledge in that area.

flipmokid12:04:52

Yeah, that's what I had created for a previous project but wanted to try and avoid it if possible. Okay, thanks very much @UJRDALZA5 for your time. It's much appreciated. I'll keep going with a Java class invoking my clojure code instead of trying to create someting in Clojure which Azure can work with directly

flipmokid12:04:03

Yeah so I have

(defprotocol AzureLambdaFunction
  (onCall [^HttpRequestMessage request ^ExecutionContext context]))

(deftype CalculationFunction []
  AzureLambdaFunction
  (^{FunctionName "TestFunction"}
   onCall
   [^{HttpTrigger {:name "req" :methods #{HttpMethod/GET} :auth-level AuthorizationLevel/ANONYMOUS}}
    request
    context]
   (.build (.body (.createResponseBuilder request HttpStatus/OK) "Hi"))))

hindol12:04:15

To understand what is happening.

flipmokid12:04:34

That's great! Thank you

hindol12:04:15

If nothing works, the functions interop code can still become a library other Clojure projects can depend on. That way, the Java bits are moved out of the way. Maybe you can create that lib!

flipmokid12:04:00

That sounds good to me! Thanks again @UJRDALZA5, much appreciated

hindol13:04:41

No problem at all. I am now intrigued as well and I will play around with deftype now.

hindol19:04:25

I have a feeling gen-class will do what you want, not deftype.

hindol21:04:22

I was so intrigued by the problem you are trying to solve, I was experimenting on my own. I have managed to generate the right signatures. Still need to figure out how to deploy the jar to Azure Functions without using the Azure provided Maven/Gradle plugins. https://gist.github.com/Hindol/78e9aa0147b6ebffe7b82d2c3fe99fc1

eudis12:04:28

Morning everyone. One of the most annoying things is adding to the end of an expression as you're coding ie for:

(def my-list []
  [[:one]
  [[:two]]
  [[:three]]])
If I wanted to add [:four] to the my-list vector above, I would manually put the cursor in front of the last ] char and hit enter, then start typing. I would have to know exactly which character to do that, even for longer lists. Is there any way to make this a bit easier? Any editor configurations, paredit config, etc. Or is this just the way that it has to be.

j12:04:46

I go to the start of vector '[' then go to the closing ']' with shortcut, then hit left arrow and type [:four].

eudis13:04:40

interesting

eudis13:04:53

Thanks I'll look at adding this to my shortcuts

manutter5112:04:32

I put the cursor in front of the first [ on the last line and then hit Option-cursor-right, and that jumps to the right spot

Kevin12:04:33

Maybe something like Parinfer would make that easier for you? You'd just have to add the new vector item on the same indentation: https://shaunlebron.github.io/parinfer/

šŸ‘ 4
manutter5112:04:47

Iā€™m using Cursive on a Mac, but Iā€™m thinking your editor probably has some kind of ā€œskip over expressionā€ equivalent.

eudis13:04:21

I'm using cursive on mac as well. I think these are good suggestions, thanks guys. I'll try them out and see how it feels

practicalli-johnny14:04:25

Clojure editors should have commands / shortcuts to jump around the code, symbols, expressions, parens, etc. It's a big advantage of using a language syntax based on a structure, that structure can be used for a great many ways to navigate and manipulate the text. There are some Cursive examples here https://cursive-ide.com/userguide/paredit.html

borkdude14:04:04

small question: in this example: (let [^Class this this] (proxy-super method param)) can the this binding be regarded as (implicitly) used, because of the proxy-super? I think so, after reading the documentation, but maybe an extra pair of eyes doesn't hurt

nonrecursive15:04:01

hey yā€™all, Iā€™m trying to pretty print this map:

{:a/b "hi"
 :a/c "hi"
 :a/d "hi"}
The result is
#:a{:b "hi", :c "hi", :d "hi"}
Iā€™d like it to print as itā€™s displayed in my first snippet, without the keyword namespace grouping (Iā€™ve forgotten what this is called?) Is there any way to do that?

Alex Miller (Clojure team)16:04:26

Bind *print-namespace-maps* to false

nonrecursive03:04:22

thanks šŸ™‚

nonrecursive15:04:16

awesome, thank you!

victorb16:04:13

hey folks, remember coming across something made in Clojure where you could define "immutable" infrastructure code in Clojure and it also worked well with the repl, but can't find anymore. Anyone know what I'm talking about?

noisesmith16:04:02

what do you mean by "infrastructure code" here? generally anything that works in clojure works in a repl

victorb16:04:34

it was functions that handles setting up/provision/down instances

victorb16:04:39

part of some lib

victorb16:04:24

hm, no, I remember looking at pallet before but then just a week ago found a different lib...

victorb16:04:37

ah, found it! https://github.com/epiccastle/spire Misremembered the scope though, it's just for the provisioning

victorb16:04:39

thanks y'all!

lukasz16:04:14

Ah yeah - I need to check out Spire - looks very promising, and Pallet is definitely not maintained anymore

victorb16:04:46

yeah, the idea behind pallet is pretty sweet though, would be great to have something like terraform but easier to play around with (give-me-the-repl-please)

šŸ”„ 4
lukasz16:04:49

I'm just wrapping up a 2 month long project where I worked pretty much exclusively with Terraform and I'm not sure if having a repl where you can make changes to the infrastructure is a good idea :-) state drift from your declarative configuration of your infra is a huge pain. Also, Terraform has a console option which sometimes allows you to poke around things in a read-only way

victorb16:04:56

the terraform console is a joke though and won't connect to your editor. I think having both is optimal, allow me to experiment in a different environment until I find what works, then I can codify and check that in (kind of like any clojure development really)

coby16:04:16

I thought maybe you were referring to this https://github.com/StediInc/cdk-clj

coby16:04:23

^ in alpha and it's specific to AWS cloudformation, but pretty neat anyway. There's a pretty cool Conj talk about it called "Goodbye YAML."

victorb17:04:43

oh, I see. Interesting, probably has some nice ideas. But being tied to AWS is a no-go for me (at least for the project I'm currently working on)

victorb17:04:36

coincidentally, this hit HN today: https://www.pulumi.com/ Seems to be talking with the language runtimes over gRPC, so adding Clojure should be possible https://www.pulumi.com/docs/troubleshooting/faq/#how-can-i-add-support-for-my-favorite-language

šŸ‘€ 4
coby17:04:39

> being tied to AWS is a no-go for me same. But I like the idea of spec having my back

lukasz17:04:45

Interesting - I guess same approach could be taken with Terraform - since it accepts both HCL and JSON, writing something in Clojure to generate appropriate JSON structures and invoking the plan+apply combo would also work.

zane17:04:14

Are there coarse guidelines for when it's appropriate to reach for records vs maps these days? I get the sense that records have fallen out of fashion, but it's not clear to me whether they're fully discouraged.

manutter5117:04:35

Records are useful when you want to dispatch a multimethod based on type. Iā€™d say itā€™s not so much that theyā€™re discouraged, itā€™s more that theyā€™re a specialized use case.

šŸ’” 4
ā˜ļø 4
noisesmith17:04:03

if all you need is type dispatch there are other options too

user=> (type (with-meta {:x 1} {:type "foo"}))
"foo"

zane17:04:12

I think I'm experiencing some decision paralysis when it comes to "type"-based dispatch. There are many options! Some that I've considered: 1. Records + dispatch via a protocol 2. Maps with a :type key, dispatch via a multimethod on :type 3. Maps dispatch via a s/or clojure.spec + multimethod on the tag returned by conform I'm curious how folks tend to choose between these options (and others!).

jumar03:04:06

I highly recommend reading this little bok by Paul Stadig: https://leanpub.com/clojurepolymorphism It's very short and discusses various approaches on a couple of real examples

šŸ‘ 4
ghadi18:04:58

there are other options, like :extend-via-metadata protocols in clojure 1.10

ghadi18:04:20

gives you per map polymorphism

ghadi18:04:34

(well, not just maps, but most commonly maps)

vlaaad18:04:31

per IObj polymorphism

vlaaad18:04:42

doesn't sound as impressive šŸ˜„

šŸ˜„ 4
zane18:04:49

Thanks! Forgot about :extend-via-metadata.

zane18:04:27

Per-map polymorphism seems distinct from the others. A bit like reify.

vlaaad18:04:51

it is, but with value/equality semantics retained

mikroskeem20:04:10

hey, can anyone suggest me a sidekiq-like library for clojure?

šŸ’” 4
nick11:04:32

hey @U0JEFEZH6 Thanks for your answer. I was interested in the same question as well. I've used sidekiq quite a lot in my past (Ruby) projects. I guess @U011YV400HX is coming from Ruby background as well? I would rather not risk using Carmine in production project because it is mostly a product of one person and it could be abandoned in the future(sorry but I've see too many cases like this in the past). If I were to choose a message broker for a small-to-mid production project I would choose Rabbitmq. It has great performance, monitoring, config options, retrying, dead-letter queue. Basically everything that Sidekiq Pro offers + much more. And it doesn't cost a fortune - e.g. CloudAMQP is free for 1mln requests per month.

mikroskeem11:04:50

i'm not coming from ruby background, i saw that discourse utilizes sidekiq and thought my (private) project could have something like that as well. couldn't come up with better description than "like sidekiq" either šŸ˜„ red-ghost seems pretty neat, thanks lukasz!

lukasz13:04:32

@U0113AVHL2W yeah, I wouldn't use a redis based queue system either, that's why my team wrote https://github.com/nomnom-insights/nomnom.bunnicula - we're processing hundreds of thousands of jobs every hour with it :-)

lukasz13:04:14

@U011YV400HX if you don't need multiple instances of an app, you might just use Java's own concurrent queues or even core.async - no redis necessary. There's also stuff like https://github.com/Factual/durable-queue

nick13:04:03

@U0JEFEZH6 https://github.com/nomnom-insights/nomnom.bunnicula looks really great! I think I'm going to use it in my next project

mikroskeem13:04:47

durable-queue looks neat! however i'd like some other queue storage method. i found few articles about postgresql & its "SKIP LOCKED" clause, might hack something together myself in some time

mikroskeem13:04:11

...since my project dances around postgres mostly right now

lukasz13:04:41

@U011YV400HX if you want to contribute: https://github.com/lukaszkorecki/taskmaster it uses 'SKIP LOCKED' and 'NOTIFY'. It seems to work, but I didn't use it in production (yet)

mikroskeem14:04:09

looks good, starred.

olttwa13:03:32

@U011YV400HX @U0JEFEZH6 @U0113AVHL2W The Clojure ecosystem now has a reliable & versatile background processing library #goose https://github.com/nilenso/goose

olttwa13:03:18

If you have a need for it, do give it a spin and ping in #goose for any issues or feature requests.

lukasz15:03:01

Thanks! I'll check it out, but FWIW I built a background job system on top of Postgres, supports retries, scheduled jobs and has an acceptable throughput (1000 job/s). I might open sauce it at some point, it pairs nicely with a cron-based (with support for extension added by Quartz to allow for seconds to be used in intervals) scheduler that my team uses that works around issues in j.u.c.ScheduledThreadPoolExecutor Also, my team maintains our own RabbitMQ framework - https://github.com/nomnom-insights/nomnom.bunnicula

mikroskeem15:03:31

Sure many things have changed within last 3 years... will check it out and might even get back to Clojure ecosystem šŸ˜„

mikroskeem20:04:20

i'm aware that there's farmhand-clj, but sadly its licensing does not fit for me

daniel.spaniel20:04:32

Is there a way to prewalk a tree/map and walk all nodes except for one particular node I want to stop the walking .. but keep the rest going? Its like saying .. I want to descend tree but when certain child is hit then stop just that traversal for that child

phronmophobic20:04:28

you could probably do this with tree-seq

noisesmith20:04:50

if you look at the code for clojure.walk/walk and clojure.walk/prewalk these are fairly tight functions, you could copy/paste and modify walk to ignore certain data, and then make a prewalk that uses that instead of the original walk

āž• 4
noisesmith20:04:23

what you want would literally just be an extra branch in the cond form in walk

daniel.spaniel20:04:33

hmm .. let me try that ( the extra cond )

daniel.spaniel20:04:19

not sure how I would do that with tree-seq though ?

daniel.spaniel20:04:27

since tree-seq spews out all the nodes .. and i want to stop spewing at one particular node ( unless I don't understand tree seq well enough )

phronmophobic20:04:06

sorry. i missed that you were transforming the data with prewalk šŸ˜¬

noisesmith20:04:11

tree seq takes an arbitrary predicate to decide if something has children and an arbitrary function to get the children

zane20:04:15

In your implementation of children you'd test for whether you're looking at the node you want to ignore and return an empty sequence if so.

noisesmith20:04:21

this doesn't help you put the whole thing back together though

4
daniel.spaniel20:04:16

i think that is problem with tree-seq .. i need the tree to stay in same shape and not be smooshed into list of nodes

zane20:04:26

Got it. tree-seq probably isn't what you want then.

phronmophobic20:04:46

yea, not sure if thereā€™s a builtin. making an alternative version of clojure.walk seems reasonable. https://github.com/redplanetlabs/specter can do this, but is an extra dependency. depending on if youā€™re doing other similar operations, you could also use clojure.zip, https://clojure.github.io/clojure/clojure.zip-api.html

daniel.spaniel20:04:25

specter is pretty nice i will grant you , but I trying not to use it and do core library only for this one

šŸ‘ 4
daniel.spaniel21:04:53

I ended up doing this ( so thanks for the encouragement to look at walk again @U051SS2EU

daniel.spaniel21:04:59

(defn copy-walk [f form]
  (w/walk (fn [n]
            (if (and (vector? n) (= (first n) :invoice/product))
              (copy-walk identity n)
              (copy-walk f n)))
    identity
    (f form)))

daniel.spaniel21:04:08

this walk will start prewalking and stop when it hits a leaf i dont want walked and just return that leaf as it is ( unwalked ) and keep on with rest of traversal

noisesmith22:04:47

oh, so it's a fork of prewalk, cool that's more parsimonious than forking both walk and prewalk

daniel.spaniel20:04:08

I've been puzzling over this for a long long time, and wonder if there is some super simple thing I am missing

Spaceman21:04:00

Hello. In the Luminus template, what is the best way to specify development and production keys?

Spaceman21:04:17

Is it by putting them in project.clj using env? I

Spaceman21:04:03

I'm inquisitive because there is also an env directory in the Luminus template, and using that might be a better way?

Spaceman21:04:30

But I'm not sure how to use the env directory to store my production and development keys.

Spaceman21:04:54

I couldn't find any documentation on that on the Luminus website

Spaceman21:04:28

especially because I'm using heroku for production. How will the production keys work then?

Spaceman21:04:09

There's a config.edn for development and production both. I suppose the environment variables go in that. But does that mean that I don't set the environment variables on heroku itself? Are there any drawbacks to doing that?

seancorfield21:04:55

@pshar10 Do you have the Web Development with Clojure book? That goes deep into Luminus and might have the answers. Otherwise, there's a #luminus channel that might help.

Spaceman21:04:14

@seancorfield I don't have that book. Is there a link to sample or a free version?

andy.fingerhut21:04:31

There are a few excerpts available on this page describing the book: https://pragprog.com/book/dswdcloj3/web-development-with-clojure-third-edition

Spaceman21:04:40

It looks empty in #luminus, so I'll try here again. I tried this but it didn't work. {:foo "bar"} in config.edn and then using it like (:require [myapp.config :refer [env]]) and then (:foo env). Is this the correct usage of the environment variable?

dpsutton22:04:48

@app.config/env yields > #object[mount.core.NotStartedState 0x5a288cca "'#'app.config/env' is not started (to start all the states call mount/start)"]

dpsutton22:04:36

once you call start it resets app.config/env to be a hash map

dpsutton22:04:46

weird system ĀÆ\(惄)/ĀÆ

dpsutton22:04:50

ah that's just mount

Spaceman23:04:36

Does it work for you or not? When I do [myapp.config :refer [env]] in a cljs file, I get the env is not defined

Spaceman23:04:13

this https://luminusweb.com/docs/environment.html says that environment variables are managed by the cprop library.

Spaceman23:04:32

But cprop doesn't seem to work in cljs

Spaceman23:04:50

does that mean that I can't unify the environment variables across clj and cljs?

Spaceman23:04:18

Because in cljs I get

The required namespace "" is not available, it was required by "cprop/tools.cljc".
"clojure/java/io.clj" was found on the classpath. Maybe this library only supports CLJ?
when requiring the cprop library

Spaceman23:04:22

so yeah, [myapp.config :refer [env]] and using (:foo env) works

Spaceman23:04:34

But only for clj and not for cljs

dpsutton23:04:58

Correct. Thatā€™s clj only

Spaceman23:04:18

So how to unify these variables for both clj and cljs?

Spaceman23:04:37

Like I use a url in both clj and cljs files which changes from localhost to the live url

Spaceman23:04:17

so that url is best placed as an environment variable, right?

Spaceman23:04:03

And then there are production and development keys too, some used in cljs files and others in clj files

Spaceman23:04:47

some overlap, others don't

dpsutton23:04:53

I think what we have at work is a map from environment to info in a cljc file. And the backend and front end can each figure out their environments from different areas but then once thatā€™s known they can reach into the map

dpsutton23:04:20

The backend can ask an environmental var or something and the front end can inspect the url