This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-02-23
Channels
- # admin-announcements (1)
- # announcements (1)
- # beginners (222)
- # boot (210)
- # cider (26)
- # cljs-dev (50)
- # cljsrn (19)
- # clojure (243)
- # clojure-art (12)
- # clojure-finland (1)
- # clojure-poland (43)
- # clojure-russia (46)
- # clojure-sg (13)
- # clojurescript (60)
- # core-async (14)
- # css (11)
- # datomic (9)
- # devcards (9)
- # dirac (2)
- # editors (13)
- # emacs (5)
- # euroclojure (1)
- # events (3)
- # hoplon (76)
- # immutant (10)
- # job (1)
- # jobs (2)
- # keechma (1)
- # ldnclj (33)
- # lein-figwheel (1)
- # leiningen (20)
- # luminus (26)
- # mount (31)
- # om (105)
- # onyx (56)
- # parinfer (29)
- # perun (12)
- # proton (1)
- # re-frame (14)
- # reagent (5)
- # sydney (1)
- # yada (15)
Just posted a question about transducers and their reducing functions: http://stackoverflow.com/questions/35566494/how-many-reducing-functions-do-collection-values-pass-through-in-a-call-to-trans Been confused about this for 2 days now 😢
It's the latter case and there are no intermediate collections.
the way transduce works is you have a reducing function f like (fn [accum value] ...) and you have a transform (called xform in the doctsring) which is sort of the transducer
transduce applies the transform (xform) to the reducing function f, which then returns a new reducing g, and then the collection is reduced with g
so the xform isn't a transformation of data, but a transformation of the function used to reduce the data
so the map transducer is something like (defn map [f] (fn [r] (fn [accum value] (r accum (f value))))) if I recall
so 'f' is the function being mapped, 'r' is a reducing function, and it returns a new reducing function
so when you call (map inc) you get back a function that takes and returns a reducing function
Heyas. Is there a macro/lib for a conditional map syntax? kinda like.. . {:a true :c false} -> {:c (binding)}
I'm doing tons of data munging and this would make my code a lot cleaner than the scattering of assocs/dissocs
hey @bradford what would the intended semantics be?
@jonahbenton: basically, it only binds the symbol on the left to the form on the right if right evaluates to true
then I can just have a big {:map} in my code of optional expressions, instead of lots of nested ifs, assoc’s, and dissocs
Hi everyone, not sure what's best practice to post a longer question here so I wrote it up in a gist: https://gist.github.com/khepin/d995f8ebe455b103fac1 I'm having some perf issues and would love any pointer to what I'm missing / doing wrong here
user> (defn g [{:keys [a b c]}]
(match [a b]
[true true]
,,(println "C was " c)
:else
,,(println "Didn't match!")))
#'user/g
user> (g nil)
Didn't match!
nil
user> (g {:a true})
Didn't match!
nil
user> (g {:a true :b true})
C was nil
nil
user>
or somethingthe tricky bit is that core.match doesn't really have a notion of free variables to be bound within patters (at least that I know of) so I had to separate the destructure from the conditional.
To do better than this you'd have to have some way to encode a "free variable" in a pattern match structure which does not conflict with referencing a variable bound value. For instance [{:a true :b true :c c}]
doesn't work because even if c isn't a closed over symbol it is interpreted to be a symbol constant rather than a binding to be established.
Reading the overview [1] it looks like you should be able to say [{... :c (_ :as c)}]
but that seems to be interpreted to mean that the kv pair [:c Anything]
exists within the matched/destructured map which isn't true.
Apologies for incoherent message. Stuck on cattle train into London :-)). Not a dig at OOP
Using stuart sierra's component
lib, is it possible to selectively restart an individual component (not the whole system)?
@alexisgallagher: In response to your question about doing a side-effect in an atom, if you want to do a side-effect, instead use a ref and send that side-effecting function to an agent inside the ref. Only when the transaction is successful (doesn't have to be retried) does the send actual get sent, otherwise it's queued in an 'outbox' which gets purged if the transaction has to retry. This way you can ensure that your side-effects are performed once and only once
I used to think this was true for atoms, but my recent experiments have show it's only true for refs.
@alexisgallagher: reading a bit more of the thread... I think you have to be aware that functions applied to atoms and refs are potentially retried, many times. If you're OK with that, go ahead and call your side-effect synchronously. If it's a side-effect like (launch-missiles!)
you may be ok, if it's (send-email!)
please ensure I'm not on your mailing list
I guess you could use add-watch! to mitigate the retry issue (and get acces to old val), but that's might be adding hair on a already very hairy solution
@malcolmsparks: bummer!
that would be kind of useful, for example if one component takes a long time to load (think a cache, remote db cx)
re: yesterdays discussion about ssl, has anyone got letsencrypt working for their jetty/whatever server? I’m about 50% there (I think) using jetty 9 https://letsencrypt.org
That seems like a useful pattern for interactive dev:
(defn handle* [req] ...) (defn handle [req] (@#'handle* req))
to make sure a function is reloaded on each run (rather than remembered in ring
's route list)Does anyone use something like this? Maybe extracted the pattern as a macro?
@pesterhazy: @boycott I only was looking into certificate generation, not actual usage with something like jetty
I have got letsencrypt running on an nginx server, much more straightforward than the whole key, pem, use these two files for this server, these two catted together for this one craziness.
@boycott: if you do manage it, write it up as a blog post
hey clojurians, i could need some help with the following code snippet
(defn auth-user [credentials] (let [user (store/find-user-by :name (:name credentials)) unauthed [false {:message "Invalid username or password"}]] (if user (do (println user) (println (str "user name:" (:name user)) ) (if (hs/check (:password credentials) (:password user)) ;;[true {:user (dissoc user :password)}] unauthed) unauthed))))
when i print the user, everything is fine
=> ({:name ben2, :email <mailto:[email protected]|[email protected]>, :password bcrypt+sha512$1b5c6807d704b5dc0d59f9e5$12$2432612431322472386731364c726252323956596264736f7556673965364551674c6a44624f537a69305a58544f6f7236495a3536576f3350497753, :user_roles ()})
but (:password user) return nil
returns
as you might have noticed, I am a newb
I'm a newb too, but does (:name user) => "ben2" as you would expect?
return nil to
that's probably because you have a map inside a seq
try (:password (first blah))
sh**t, see it now
ther are parens around the map
yeah, db gave you a list of stuff
I make that same mistake sometimes
Any Expectations test people around? I see odd behavior, where it appears an (expect) is being done out of order. I have an (expect ...) line before a line that changes the DB, but the expect fails because it appears the expect is actually happening after the db change
now it's working 😉 thanks a lot.. problem blindness 😉
yup! cheers
@dev-hartmann: looks like your map is wrapped in a list.
@karl: no biggie, thanks anyway 😉
hey @raphael recur is to be used from the "tail position" in its containing form- tail essentially meaning last. the idea is that the containing form gets turned into a loop, and recur's location determines where the loop ends. so a line of code that appears inside the containing form but after recur presents confusing semantics. e.g. in (fn [] (recur) "xxxx") the containing form is the fn, and "xxxx" is in the tail position. recur wants to loop back up to fn, but the "xxxx" is still sitting out there- inside the form but outside the end of the loop. so the compiler will complain. does that help?
in fact I don't undertand when the compiler take care of what there is after the recur
hey @pesterhazy you actually can restart individual components in a system- but in doing so you get a different instance of the component than exists in the system map or in dependent components.
@jonahbenton: can I patch that back in into the system afterwards?
sure, the system map is just a map
but in general the semantics aren't intended to be the granular sort where you may have a backoff retry or other machinery. at the component level it's more coarse grained.
@raphael can you say more? the reason recur exists is that there are 2 things that can happen within the containing form at "tail position"- either you loop back up, or you don't, and the value of the last expression- in tail position- becomes the value of the containing form. so you need to tell the compiler which one to do- (recur) means loop, and the absence of recur means fall through and exit the loop
sorry- misread
recur is a compromise solution for problems that in other languages may be solved with recursion
the jvm does not support tail recursion
recur is like goto
yes I think I understand thank a lot @jonahbenton
sure, good luck
Hi guys. I'm trying to understand how can one tweak clojure.core/*compiler-options*
in leiningen project. What exactly am I trying to do is to allow direct-linking
but no luck so far.
Thank you, @mpenet I tried this on simple app
(ns zippy.core
(:gen-class))
(defn f1 []
(println "f1 impl 1"))
(defn f1 []
(println "f1 impl 2"))
(defn -main
"Start a production system."
[& args]
(f1)
)
Correct me if I wrong. This code should print f1 impl 1
if direct-linking
is enabled, but it prints f impl 2
I dont' really know how it works internally, but my guess is that only runtime redefs would cause a first f1 call
All I know from clojure docs and changes.md is that runtime function overriding won't work. So I wonder what is the best way to check that direct-linking is really enabled. This code is the only idea I came up to..
I'm trying to benchmark my application and see if this allows any performance improvement, but no difference so far
> well perf difference is barely noticeable in practice I understand that, but I need numbers
as far as I understand direct linking, printing “f impl 2” is expected
yes, thank you, @joost-diepenmaat: it's my bad, I misunderstood what exactly runtime overriding means
good it’s working as expected
I personally don't know of any benchmark that shows a noticeable benefit. If you find any please let me know
we are only concerned about performance at the moment, we optimized a lot of stuff already, so we are looking for some new ideas
hey @edvorg what does a profiler say about where your bottlenecks are?
@jonahbenton: We optimized mostly everything we could see in profiler. Now we have some external api cals and some heavy data transformations we can't get rid of. We refactored code with transducers where it's possible and got small improvement from this. So what I'm looking for are some unusual optimization paths that can't be seen in profiler
@malcolmsparks: thanks. Yup, I was aware that no side-effects were allowed inside the atom's update func, and that that function might be retried. What's puzzled me mainly is how to trigger a side-effect only when the update function actually returns a changed value. I had this solution using dosync and refs:
defn transform-item [x] ...)
(defn do-side-effect-on-change [] nil)
(def my-ref (ref ...))
(when (dosync (let [old-value @my-ref
_ (alter! my-ref transform-item)
new-value @my-ref]
(not= old-value new-value)))
(do-side-effect-on-change))
But what made me think I should be using an atom instead of refs is the catechism to the effect that refs are for coordinated state changes and I'm really only updating one thing here.
But in the end it seems like refs are the best way to go, whether I return a value from the dosync to report the change or use an agent if I want that fact asynchronously.
@mpenet: I don't want the watch triggered on every change, only by changes due to this snippet. Couldn't figure out how to scope a watch within a transaction.
then you'd have to do this manually with an atom: using compare and swap, check it's return value and act when the swap occured.
But I think that fails because then I don't have access to the old and new value from within the transaction, do I? I may be missing something obvious here...
alexisgallagher: https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/compare-and-set!
@alexisgallagher: you can see one of the issues through the snippet above. the value of my-ref can change in between in the let assignment to old-value and the alter!
if I recall correctly, because the transaction is altering the value of the ref, it can't
right- not within the transaction, but in the context of the wider world
and the value of my-ref after the alter may not be the final value for the ref after the transaction
clojure's stm is an mvcc
correct, but other threads may be executing their own transactions
dosync does not serialize across threads
an issue would be, if it also read another ref that it did not also write to, then if I recall you need to use ensure on that ref
right- not within a single transaction- the view of the world is fixed. but another thread may complete a transaction against the same ref
when it comes time to commit, at the end of dosync
once the transaction completes, that code returns true or false indicating if the transaction actually changed the state of the ref
right- so my point is- within a transaction, you may detect a successful change, but that's different from reporting change across transactions
but, that is, as far as I know not what is the ask here, this is "hey, I am updating this state conditionally, and want to run action A if it updates"
the twisted and complicated ways people end up doing that is crazy, there is a built in function that lets you write your own swap!
with whatever behavior you require
yeah- i was suggesting agents- e.g. enforce explicit ordering of the incoming message stream- because reporting on change across threads is not what atoms and refs solve for, and reinventing the retry logic is probably undesirable.
iow, just use a queue
sure, but you are definitely serializing everything then, and you have to sort of invert the control, the agent has to execute the side effecting stuff, so you aren't reading values from identities at that point, you may have to restructure your code to be more actor like
sure- the code snippet above can be handed directly to an agent- receive a message, transform agent state, if a change occurred fire off the notification- but if you have other logic to also execute following the change, or if you have to use await- then it's not a good pattern.
if the system is actually request-response, not change-notify, the pattern doesn't fit
I thought there was a nicer error message for missing the parameter vector these days
well, it came back with a java.lang.IllegalArgumentException: Parameter declaration "when-let" should be a vector
IllegalArgumentException Parameter declaration "when-let" should be a vector clojure.core/assert-valid-fdecl (core.clj:7111)
ska: well when defn sees a list there it assumes you are using the multi arity version of fn, like (defn f ([] 1) ([x] x)), in which case the when-let there should be a vector
Just imagine that tiny snippet to be a longer function with a real body in the when-let and a longer function name and all... It was just when I cut everything else away, that I realized my stupidity.
Anyway, thanks for listening. Feeling better now. Even better than M-x doctor. 😉 Back to work again.
Now I'm confused again. Are you two concluding my dosync approach is incorrect? @hiredman @jonahbenton
alexisgallagher: not exactly incorrect, but I think you would be much better off using an atom and compare-and-set!
Hmmm. Not clear to me how to get the same result with that, given that I'm actually starting with an update function. Wouldn't that require doing locking
around the compare-and-set!
?
So I build my own retry loop?
e.g. you can make it return true/false based on if your business logic actually changed the state of the atom
Hi everyone, I'm just starting to use clojure and doing an exercise I came across an error I just don't understand, maybe someone can help me
Is that :b#something keyword illegal for some reason? It doesn't seem to be outside of a let
I'm a little surprised that works? From http://clojure.org/reference/reader : "Symbols begin with a non-numeric character and can contain alphanumeric characters and *, +, !, -, _, ', and ? (other characters may be allowed eventually)." ... Keywords - Keywords are like symbols, except: They can and must begin with a colon, e.g. :fred. They cannot contain '.' or name classes. Like symbols, they can contain a namespace, :person/name A keyword that begins with two colons is resolved in the current namespace: In the user namespace, ::rect is read as :user/rect"
I guess it could be related to the version of clojure used too, sometimes what the reader allows in symbols changes, but I don't recall that being one of the things that has changed
some libraries that used to be popular actually used '#' in keywords as part of their dsls, and if the reader suddenly clamped down on that stuff it would break things
there is at least one jira issue that is sort of like "figure out what to do with this mess"
'>' is actually used in the names of the functions clojure generates for defrecords (a more recent example of clojure using characters outside of the ones listed there)
@nkraft nope but I'd recommend https://github.com/mpenet/alia is a bit more up to date and used by some pretty large projects now (disclaimer: I authored it)
Thanks. I actually gave up on Cassaforte and switch to Alia, which looks pretty good with Hayt.
Yeah the dsl is identical with hayt, cassaforte actually used hayt internally until recently
@mpenet: While I have your attention (more or less), how does one get pooling options to work? I've been trying things like :pooling-options {:core-connections-per-host [:local 16 :remote 16]} but I get unpacking and nth keyword errors.
I hope I would need to use Cassandra because those library names are great @mpenet!
@juhoteperi thanks :)
@mpenet: Yup, that was it. Problem with looking at old examples, I guess. Got the pooling working, Thanks!
If you need to push for perf and dont care about lazyness you can also play with :result-set-fn opt in execute, you can hit IReduce path and have a bit more control over what you get in return and avoid seqs
I'll put that on the list of things to play with. I need to push several billion entities to a Cassandra datastore. Fun, fun, fun.
What webservers do people like to use? I'm seeing immutant, aleph, http-kit, etc., but I'm not really sure what the pros/cons of each are
They had a great deal at the start of the year for a personal license for $100-$200 but I think it was a temporary thing