Fork me on GitHub
#clojure
<
2019-12-17
>
caio03:12:02

What's the proper way to stop an agent from receiving new commands?

seancorfield03:12:50

@caio Can you explain the problem you are trying to solve? Since requests sent to agents are queued, how would you want the agent to behave in this case, and how would you expect additional send or send-off calls to behave?

caio03:12:35

Might be just overly defensive programming from me, but the use case is a stuartsierra.component that uses agents internally to manage state. I wanted to make sure that on stop the agents stop receiving messages and then wait for every agent to finish processing the existing messages and then return. I know that by the time stop is called on this component, there shouldn't be any new messages about to arrive, but I want to be extra safe

potetm03:12:22

Agents are run on a global pool. They can only be stopped with shutdown-agents.

potetm04:12:16

iow - it doesn’t play well with components

potetm04:12:43

You can consider managing your own executor and use java futures.

caio04:12:03

Hm. Ok, makes sense. Thanks

pablore04:12:50

Any tips on finding cyclic dependencies?

vale04:12:33

doesn't the compiler find them for you?

pablore04:12:34

It’s a big project, I haven’t required all the files. Maybe I should try that

pablore04:12:08

The exception happens when I run cloverage on my project. But the stack trace doesn’t show where the cycle is

borkdude09:12:12

@ Do you mean between namespaces?

nickik09:12:30

What are peoples tips on creating docker images. I have started again with clojure and getting into the whole deps.edn lifestyle. I used one of these packaging tools to build an uberjar. How do I now build th image? Should I just do that total outside of clojure tools?

kwladyka10:12:07

building an image is ci/cd job. If you use github use github actions. At least it is me recommendation.

kwladyka10:12:57

name: master -> docker

on:
  push:
    branches:
    - master

jobs:
  build-docker:
    runs-on: ubuntu-18.04
    steps:
    - name: checkout
      uses: actions/[email protected]
      with:
        fetch-depth: 1

    - run: docker build . -t {{ github.repository }}/golem:latest

    - run: docker login  --username ${{ github.actor }} --password ${{ secrets.GITHUB_TOKEN }}

    - run: docker push {{ github.repository }}/golem:latest

    - name: Slack notifications
      uses: 8398a7/[email protected]
      with:
        status: ${{ job.status }}
        author_name: GitHub Actions
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
      if: always()

kwladyka11:12:26

of course you have to prepare Dockerfile

kwladyka11:12:06

and yes, build image outside of Cloure. Building image using lein build-image or something like that is not how it should be

kwladyka12:12:10

At least it is my opinion. All kind of compilation of third party code or whatever shouldn’t be done by lein - so that is why I like deps.edn . I see developers without DevOps experience put this things into what they know, so in Clojure only because they can. But it is not the right approach. It should be done by ci/cd solutions. Otherwise project.clj do job which shouldn’t do, it is complex, run things with assumption you have something installed on your localmachine to run REPL etc.

kwladyka12:12:26

right tool for the right job

kwladyka12:12:51

lein in my opinion should be use to Clojure and nothing else

kwladyka12:12:32

project.clj and lein should be only about application, not about infrastructure etc.

kwladyka12:12:59

the same for deps.edn

kwladyka12:12:25

But yes it is my personal strong opinion in this topic 🙂

p-himik12:12:48

Yeah, I agree about not adding anything Docker-related to project.clj or deps.edn. But I'm not sure it's bad to build images locally if you have a proper Dockerfile.

kwladyka12:12:52

Do you mean build image locally and push it to docker hub?

p-himik12:12:48

I now realize that I might've misunderstood the initial premise. Yeah, docker hub or wherever you have to deploy it.

kwladyka12:12:27

yes, probably we misunderstood 🙂

kwladyka12:12:16

you can, but it is always the place to make a mistake when you are tired. Push the image which you did temporary for debug purpose or experiments.

kwladyka12:12:38

While you back to project after half a year you have to remember what to do to deploy

kwladyka12:12:56

you can be tired or feel sick this day

kwladyka12:12:37

I always do ci/cd and I even run dev environment in docker-copmose.yaml instead of brew install foo

kwladyka12:12:05

It makes me feel comfortable everything work everywhere

p-himik12:12:48

I'd say just take a day off in that case. :D Not being able to concentrate can definitely lead to errors in all sorts of places, not just when deploying/publishing an image.

kwladyka12:12:53

real life doesn’t work like that

kwladyka12:12:10

you are excited about doing something to late hours and you will make a mistake

kwladyka12:12:27

your wife will talk to you during the deploy or something will make you nervous

kwladyka12:12:39

I mean it is our nature to not be constant

kwladyka12:12:56

While doing ci/cd is so easy this days there is no reason to not do it

kwladyka12:12:16

And the best thing for me is: I don’t have to remember or trust doc about how to test or deploy things

kwladyka12:12:27

I am sure it is like it is

kwladyka12:12:40

Even after half a year

nickik13:12:09

Make sense. It requires access to the ci whenver you want to get the container.

kwladyka11:12:21

clojure -A:depstar -m hf.depstar.jar form-validation-cljs.jar -v you can build jar like that from deps.edn Do it in Dockerfile

gtzogana15:12:11

hello, suppose i start a websocket server in my repl and then lose the handle on it. is there any way of closing it, namely by somehow clearing everything running in that repl session?

vemv17:12:56

are you using component/integrant? while those don't make the issue impossible, it happens far less often

gtzogana15:12:32

i can unmap all the variables, but this won’t get it done, i don’t think

jumar15:12:52

@gtzogana I think the easiest thing is to restart the repl

gtzogana15:12:49

yeah, same. i was trying to find a way around that but there may not be one.

borkdude15:12:12

"Have you tried turning it off an on again"

gtzogana16:12:54

dad gum technology

hindol.adhya19:12:25

When you lose the handle to it, won't it be garbage collected? Sorry, not too familiar with REPL internals.

noisesmith19:12:54

it depends on whether it has a gc root (which in turn partly depends on the lib you used to manage your websocket, if any)

noisesmith19:12:25

eg. sente keeps a data structure full of websockets that are active, so its sockets won't gc until you ensure they are disposed of

vemv19:12:47

yes, keeping IO-related connections open is a good way to create a memory leak (in case you ever face that interview question ;p)

vemv21:12:45

I just noticed that my Component system is being started as part of the lein uberjar process. That doesn't seem clean, as compiling code shouldn't trigger side effects. It also occasionally breaks my CI build when some of these side effects encounter something unexpected Any idea of what could be causing this? Maybe there's a gotcha related to Clojure AOT compilation, or uberjarring etc I certainly have nothing silly in place, like a top-level side-effect or top-level invocation to (component.repl/reset).

noisesmith21:12:15

@vemv "compiling code shouldn't trigger side effects" - clojure has no "compile mode" - if something is at the top level of your file, it runs when loading the file, period

noisesmith21:12:50

so all it takes is defining the system, to create it when aot compiling

noisesmith21:12:56

sometimes you can correct this with a delay which you force via @ for usage, the cleaner solution is to create your system only inside -main, and have a helper function for the repl that you run by hand

andy.fingerhut21:12:06

If calling load-file on a file, or require on a namespace, causes some function to be called, then so will compiling the file.

vemv21:12:12

The thing is, I only create define defns. Essentially my project's namespaces only contain defns and no top-level side effects (like (do-something!)). I don't expect compilation to invoke any of these defns

noisesmith21:12:32

@vemv to narrow down what creates the system, you can explicitly print the current thread's stack trace inside the function that creates the system

vemv21:12:31

there are a few -main defns in different namespaces, so I was thinking whether uberjar could be invoking a random one

hiredman21:12:37

I bet you have a user.clj

noisesmith21:12:49

it won't invoke any -main when compiling

vemv21:12:11

I have but they are pure

noisesmith21:12:00

(Thread/dumpStack)

vemv21:12:17

neat trick, that should shed light

hiredman21:12:04

If you have a user.clj it will be loaded while compiling

vemv21:12:35

Thankfully I only have dev/dev.clj, outside of the uberjar target paths (require 'dev) fails (as it should) in prod

hiredman21:12:12

Find your call to component/start, add logging and work backwards

hiredman21:12:26

You may not know what a top level side effect is, so when you say you don't have any, that may be incorrect

vemv21:12:44

You can convey the same message without implying I don't understand certain concept :) But yeah, grepping or reasoning can always fail, so I'll Find your call to component/start, add logging and work backwards

hiredman21:12:15

I said "may" a few times there, interpret as you will

andy.fingerhut22:12:01

Definitely a bit curious to know what you find out, if you do.

vemv22:12:47

alright, I'll satisfy your curiosity (at the expense of saying something embarassing): the system wasn't being actually being run - instead some side effects were legitimately running as part of the uberjar process (which has various :prep-tasks steps) i.e. false diagnostic induced by some extraneous output and bit of a contrived build (which I authored ;p) so sorry for the noise! At least I learned about (Thread/dumpStack) (throwing an exception would have worked as well, although (Thread/dumpStack) seems cleaner as throwing an exception may fail CI for unrelated reasons, before the bug happens)

noisesmith22:12:23

also someone might have an over-eager exception handler in my experience

mark54022:12:14

Another thing I do is create and print the stack trace for an exception, without throwing it. This is better than a full dump if you have lots of threads and only need the current stack. For example:

(.printStackTrace (IllegalStateException. "debug") System/out)

noisesmith22:12:33

the method I shared above only prints the trace for the current thread as well

mark54022:12:49

oh, you're right! thanks.

noisesmith22:12:54

but sometimes (.printStackTrace (ex-info "debug" {... ....}) if you also want to pretty print some data in context

mark54022:12:24

Makes sense.

noisesmith22:12:43

(ins)user=> 42
42
(cmd)user=> (.printStackTrace (ex-info "debug" {:*1 *1}) System/err)
clojure.lang.ExceptionInfo: debug {:*1 42}
	at user$eval22.invokeStatic(NO_SOURCE_FILE:1)
	at user$eval22.invoke(NO_SOURCE_FILE:1)
	at clojure.lang.Compiler.eval(Compiler.java:7176)
	at clojure.lang.Compiler.eval(Compiler.java:7131)
	at clojure.core$eval.invokeStatic(core.clj:3214)
	at clojure.core$eval.invoke(core.clj:3210)
	at clojure.main$repl$read_eval_print__9068$fn__9071.invoke(main.clj:414)
	at clojure.main$repl$read_eval_print__9068.invoke(main.clj:414)
	at clojure.main$repl$fn__9077.invoke(main.clj:435)
	at clojure.main$repl.invokeStatic(main.clj:435)
	at clojure.main$repl_opt.invokeStatic(main.clj:499)
	at clojure.main$main.invokeStatic(main.clj:598)
	at clojure.main$main.doInvoke(main.clj:561)
	at clojure.lang.RestFn.invoke(RestFn.java:397)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:37)
nil

doubleagent22:12:17

I've got a luminus site which is giving 403 errors on one page which doesn't require a login, but it looks to me like everything is okay. The csrf is injected into the page during the GET request, and I can see the __anti-forgery-token: ... in form params on the POST. Not sure what's missing.

doubleagent22:12:45

Anyone have a clue?

vemv22:12:14

you might want to ask it over #luminus