This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-04-21
Channels
- # aws (1)
- # aws-lambda (1)
- # beginners (27)
- # boot (16)
- # cider (1)
- # clara (54)
- # cljs-dev (4)
- # cljsjs (8)
- # cljsrn (25)
- # clojure (148)
- # clojure-dev (2)
- # clojure-finland (1)
- # clojure-france (18)
- # clojure-italy (10)
- # clojure-nl (3)
- # clojure-russia (27)
- # clojure-sg (2)
- # clojure-uk (17)
- # clojurebridge (6)
- # clojurescript (70)
- # core-async (1)
- # css (6)
- # cursive (35)
- # data-science (3)
- # datomic (22)
- # events (4)
- # jobs (18)
- # jobs-discuss (14)
- # leiningen (4)
- # lumo (22)
- # off-topic (20)
- # om (5)
- # om-next (1)
- # onyx (47)
- # pedestal (107)
- # re-frame (43)
- # reagent (1)
- # ring (2)
- # ring-swagger (2)
- # rum (18)
- # sql (15)
- # unrepl (4)
- # vim (61)
- # yada (3)
delay
does the Right Thing when deref’d from multiple threads, right? Computation will happen at most once?
@tbaldridge by the way thanks for the diffing hints
everyone: has anyone tried/considered or seen something that utilizes clojure.spec to generate SQL (and more specifically SQL joins)?
(that sounds as scary as an ORM 😄 ) but I believe it can of a very good (clojure.spec) kind haha
the way I imagine it: let’s say you have a table called “products” and a table called “images”, I want to get all products with all their images joined; I can have a clojure.spec for a “product” and for an “image” and a spec that describes how they link together; from that generating something like a HoneySQL clojure map can be pretty straightforward I think
cons: it can be a leaky abstraction depending on how it’s done (I haven’t thought much about it)
let me know if you think this is a good idea and and even better why it might NOT BE a good idea 🙂
@cddr I disagree with the advice to avoid with-redefs. There's nothing wrong with it. It rebinds the root from the moment you enter its s-expression, to the moment you leave it. There's no issue with threads either, since it does so atomically. What @tbaldridge is talking about are just some caveats you need to be aware of, not actual issues with the function. For example, because the root is restored when you leave the s-expression, if you have a thread still trying to access it after you've left, it will now get the old binding back. Similarly, if you have a lazy sequence, which will try to realize itself after you have left, it will get the old binding too. This is normal, since it restores the old binding once you leave the s-expression.
I personally agree with Rich's advice, design should not be a consequence of wanting to make testing easy. With-redefs does come in handy to test things, and changing the design of the code to avoid using with-redefs is not always the right thing to do.
Now, if all your tests must use with-redefs, chances are you made some wrong choices, but if you know the trade offs, and believe your design is good, I don't see any issue using with-redefs, and I'd say that's better then forcing a change in your design for testing only
@didibus nah, avoid with-redefs
because it's good design. Avoiding uncoordinated global mutation is good design. That's what clojure is all about right? Immutable data, pure functions? Why do we throw all that away at testing time?
There are also good reason to alter var roots in prod, like for AOP style programming.
But above all that with-redefs encourages testing the wrong thing.
Let me give you an example: with with-redefs I may say: "Make sure foo
is called with 42 as an argument". That's not testing what matters. Why do I care if foo
is called?
In reality I'm subbing foo
because it's side-effecting. In which case what I really care about is that I'm creating the correct side-effect. So I really should be testing that, and not the fact that bar
called foo
.
On top of all that, if foo
is really side effecting, why is it buried so deep in my app that it's hard to swap out via means other that var redefinition? That's another design problem: side-effecting deep in the call-stack.
But really, I've been programming Clojure for many years, I've written compilers, I've written crazy macros and DSLs. And I can't get with-redefs right every time. How can I expect a mid-level developer or even a junior-level get it right?
And I'll stop ranting now 🙂
@tbaldridge Well, I agree with you on those. You should probably just have an integration test for that side-effect. And you shouldn't side effects deep in the call stack. But, if for some reasons you do, and you can't get rid of it because of time constraints, opportunity cost, or it really makes some other things easier, why not still unit test it with the help of good old with-redefs?
The cost of getting with-redefs wrong in a test isn't that high though. Your unit tests just fails.
And then you stare at it for a hour or two, as does the next guy who has to maintain your tests...
That's really what got to me over time, I'd start working on a project for a client, make a small modification, and all the tests would break. Or worse they wouldn't break. That's probably the most common problem. I'd modify a function, run all the tests and everything passes! Since the code I modified was all mocked out via with-redefs
@tbaldridge Hum, you're slowly changing my mind 😛 I might have to revisit a co-worker's code review now
Its a case where the app is a simple ETL job. It works using this model where there's a pipeline of steps which take data and meta and transform them, and return the transformed data and meta. The meta is used by later transform steps to give details based on the previous steps that can help them decide how to do their transformation. Now, some of the steps have side-effects, like logging, and pumping out audit info, metrics, etc.
Could be, I don't recommend refactoring a ton of namespaces just to get rid of one with-redef.
Well, its the case where most steps don't have side effects, and injecting and carrying all these states throughout the transformation steps would feel just like an annoyance, each of the side effecting steps might need a slightly different metric state, etc. So its like a small convenience evil, and to mock those with-redefs comes in handy
@tbaldridge Oh, BTW, kind of a parenthesis question. I've read your blog post. I was wondering if you know why Vars were made ThreadLocal? I couldn't figure out a scenario where it is useful
That's how bindings work, with thread-local:
`(binding [*out* some-stream]
....)
I know, but why ThreadLocal? I mean, it doesn't seem like its needed for the implementation. The stack frames were enough to give the dynamic extent no? Why wrap them in a ThreadLocal?
Because otherwise one thread's call to binding would clobber another thread's.
I kind of get it conceptually, but I was trying to find out an example of when that would happen and I couldn't
with-out-str is one case: https://github.com/clojure/clojure/blob/clojure-1.9.0-alpha14/src/clj/clojure/core.clj#L4668
call that from two threads at once without thread-local bindings, and you'll get corrupted text
Please do!
Just created #integrant for discussing https://github.com/weavejester/integrant
Hey fellow clojurians! Could you try this in your repl? (Clojure 1.9) and reports the repl and if it managed to evaluate it or not:
#:#_()#! bang bang
#?(:whatever 42); now a blank line
#?@(:default ())foo
{:bar :baz}
I am not sure what you want to achieve exactly. Typing #:#_()#! bang bang
in my REPL gives me a RuntimeException EOF while reading
user> #:#_()#! bang bang
RuntimeException Reader tag must be a symbol clojure.lang.LispReader$CtorReader.invoke (LispReader.java:1221)
()
user> #?(:whatever 42); now a blank line
user> #?@(:default ())foo
RuntimeException Reader conditional splicing not allowed at the top level. clojure.lang.Util.runtimeException (Util.java:221)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: foo in this context, compiling:(*cider-repl localhost*:1:7821)
user> {:bar :baz}
{:bar :baz}
user> *clojure-version*
{:major 1, :minor 7, :incremental 0, :qualifier nil}
#foo{:bar :baz} in my vanilla repl (clojure.jar + rlwrap)
in a lein repl using 1.9 alpha14 the blank line is fine, and I get a conditional splicing not allowed at top level error, and unable to resolve symbol: foo, and then {:bar :baz} as a non-namespaced map
same clojure version, so nrepl must be broken for these reader changes
(using REPL-y 0.3.7, nrepl 0.2.12)
is anyone here familiar with the elastisch library?
i've used it in the past @ben.mumford , not so much recently because it doesn't support async
good man, any idea how to get it to connect to an elasticsearch cluster?
the connect method only takes a single url (unlike the official java rest client)
what happens if the server is down?
any help greatly appreciated
@ben.mumford spandex (a lib I maintain) handles failover etc
i tried that but when i use it in my application (simple ring stuff) the number of file descriptors my application uses keeps climbing
i chose spandex initially specifically for that reason
you got a fd leak with spandex? doesn't sound normal, are you re-using the "client" instance or creating one per request maybe?
i created the client once and reused it
my application takes requests and proxies through the elasticsearch
We've been using it for quite some time on some heavy stuff and never had any leak, I am curious on what's happening here
eg: (def ^:private client (spandex/client {:hosts [es-cluster-url]})) (defn ^:private read-docs [client index type {:strs [query from size] :or {query "*" from 0 size 10}}] (let [response (spandex/request client {:url (spandex-utils/url [index type "_search"]) :method :get :body (create-query from size query)}) hits (->> response :body :hits :hits)] (timbre/info "Query:" query "From:" from "Size:" size ". Found:" (count hits) "matches.") (map map-hit-to-doc hits)))
the read-docs method is then called from one of the endpoints in the app context
@ben.mumford i don't really have the failover problem because of the way i deploy ES - i deploy a proxy (nodata) instance on every host, which then manages forwarding to a live ES data instance
i suppose it's possible for a nodata instance to die, but it's not happened in the last 5 years, which is more than i can say for the data instances
having spoken with @mpenet offline, we believe the problem was me bungling it creating multiple clients accidentally (not the spandex library). incidentally, despite my ineptitude the library has been very easy to use and is highly recommended
@mccraigmccraig we used to do it this way too, it's a good approach too. Spandex does sniffing and detects new/dead nodes and does penalisation if too many failures etc etc, one advantage is that you don't have to worry about compatibility among other things (it just uses rest)
i was going to look at spandex... we are currently using a hacked version of qbits.esearch
right - was very easy to replace the http client with our own
cool, i'll check it out - when the customers stop demanding new features 🙂
question
(defn my-func [opts]
(assoc opts :something :else))
(let [fqn (resolve 'my-func)]
;; how to use fqn to get func and call it?
(fqn {:hello :world}))
Pretty sure I got bit by this on CLJS while using web workers. with-out-str
was capturing other println
s.
oh wait, that just works
hmm. so i’m trying to serialize functions names to strings, and then unserialize them to call them
granted I'm still wearing my training wheels with clojure, but just discovered a new friend - cl-format. For kicks and giggles, added the following snippet to rosetta code as a clojure solution to the word wrap problem:
(defn wrap-line [size text]
(clojure.pprint/cl-format nil (str "~{~<~%~1," size ":;~A~> ~}") (clojure.string/split text #" ")))
cl-format...some serious lisp-fu there...and a throwback to...I don't know, the 70's?clojure -> common lisp -> lisp machine -> multics...and still kicking (edit and apparently -> BOS)
Posted in leiningen, but haven’t seen much activity there and figured this may be relevant here too (before making the post on stackoverflow or something):
I have a lein project.clj
with a repo in :repositories
configured with :creds :gpg
. I have an GPG encrypted credentials.clj.gpg
file stored under my ~/.lein
profiles dir. This works.
The problem I have is that I’m being prompted in the terminal for my GPG passphrase every time Leiningen needs to access this file, e.g. lein deps
, lein install
, lein repl
, etc.
I have tried setting up an gpg agent daemon via eval "$(gpg-agent --daemon)"
, but I am still having Leiningen repeatedly request for my passphrase.
I have looked online, but haven’t been able to find an answer. Any suggestions from anyone here?
@mikerod I have the following magic stuff in my .profile
also note that you may or may not need a :signing {:gpg-key "
entry in your project/profile
the export GPG_TTY=$(tty)
is necessary for me on OSX.
I’ve heard of the GPG_TTY
part and have that (I actually never really understood why it was needed). I’m on OSX too.
Thanks for those details though. I’ll give the rest of it a try. it is a bit different than what I’ve done
good luck. I get desparate every time it breaks 🙂
When you put a dependency in a toplevel :dependencies clause of a leiningen project.clj, what profile does it really go into? Context: I’m trying to have a project with a shared codebase where I can still have separate ubjerjars for different components that only have the deps they need.
You should take a look at https://github.com/zcaudate-archive/lein-repack which is a lein plugin that attempts to automatically solve this problem
apparently it’s http://docs.caudate.me/lucidity/lucid-distribute.html now
good to know!
I downloaded a project in clojure and I want to run it. I go to the directory and run the lein run command but I get the error "no. Main main namespace specified in project.clj."
@jsdiaz19 maybe the project isn’t runnable. Perhaps it’s a library?
What project is it?
@weavejester it is the proyect https://github.com/nbeloglazov/snakejure
@jsdiaz19 I think you need to define main in project.clj
I took this screenshot from project -->lein new app myapp
@jsdiaz19 there’s no :main
namespace stipulated in the project file, and nothing in the README about executing it. Maybe it’s unfinished?
thank you very much. Then how could I do a multiplayer game with sockets in clojure?
(char 80)
--> \P
one mpore dumb question: how do I do string -> int without calling "read-string" ? or should I just call read-string?
(Integer. my-str)
will throw an error if it's not an int
Quick question, is there an article that explains a good workflow and tools for Clojure? I use Intellij and Cursive
@qqq Long/parseLong
I'm trying out the new version of duct https://github.com/duct-framework/duct (0.9.0-alpha2) and I would like to know if it has support for figwheel. If so, how can I start the figwheel server?
If you generate the project with +cljs
a figwheel server will be included (through https://github.com/duct-framework/module.cljs). Check out the README in your generated project for instructions on how to start it.
Seems like it's configured to only reload files in the browser when you call (reset)
in the repl
I am trying to read some clojure dumped with (binding [print-dup true] (prn data-structure)) When I try to read it, it complains that it cannot find a matching method create. The dumped code has entries like so: #=(datomic.query.EntityMap/create [#=(clojure.lang.MapEntry/create
Try lein trampoline run -m clojure.main figwheel.clj (where figwheel.clj has the contents above
Hello. Are strings returned by (name :some-keyword)
somehow different from ordinary literal strings? We're have a Java/JNI API that crashes on the former but accepts the latter.
no, they are just normal Java strings
the reader interns literal strings, and I thought the keyword and symbol creating functions end up intern strings too, but I don't see where
neither of those things is true
the compiler will store string literals in the constant pool of the compiled class which has the effect of interning them I think, but the reader does not explicitly call intern on strings afaik
And now comes the inevitable moment where I slap myself on the forehead 😄
(defn my-utility-function [name] ... (name name))
Lisp-2 background...
symbols (and keywords, which use symbols) used to intern, but we found that had a significant performance cost