This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-02-11
Channels
- # adventofcode (8)
- # announcements (1)
- # arachne (23)
- # beginners (146)
- # boot (4)
- # calva (2)
- # cider (48)
- # cljs-dev (17)
- # clojure (214)
- # clojure-austin (2)
- # clojure-berlin (1)
- # clojure-europe (9)
- # clojure-italy (5)
- # clojure-nl (2)
- # clojure-sanfrancisco (2)
- # clojure-spec (124)
- # clojure-uk (67)
- # clojured (3)
- # clojurescript (95)
- # community-development (7)
- # cursive (68)
- # data-science (1)
- # datomic (80)
- # emacs (19)
- # figwheel (3)
- # figwheel-main (5)
- # fulcro (61)
- # javascript (2)
- # kaocha (1)
- # off-topic (25)
- # pathom (21)
- # pedestal (1)
- # perun (4)
- # reitit (11)
- # ring-swagger (2)
- # shadow-cljs (55)
- # spacemacs (4)
- # sql (8)
- # test-check (16)
- # tools-deps (2)
- # vim (13)
- # yada (4)
Has anyone seen the following error when running 'brew install clojure'? brew install clojure Error: No available formula with the name "clojure" Clojure isn't really a program but a library managed as part of a project and Leiningen is the user interface to that library. To install Clojure you should install Leiningen: brew install leiningen
you must be on a very old brew @jrbrodie77
@ghadi yes... turns out that reinstalling brew did the trick. brew update was also broken, which is how I ended up stuck on an older version. Didn't realize it would affect the package index.
Is it possible to define a custom implementation of toString
for defrecord
s? I have already defined a clojure.core/print-method
for the record-type in question, which helps in REPL, but is not useful when I try to generate formatted string with format
functions. Looks like format
function directly calls toString
.
(defrecord A []
Object
(toString [this] "AAAAA!!!"))
=> user.A
(A.)
=> #user.A{}
(format "scary? %s" (A.))
=> "scary? AAAAA!!!"
I'm using clojure.tools.deps.alpha
to resolve-deps
but I get:
Expanding javax.servlet/javax.servlet-api #:mvn{:version 3.1.0}
Execution error (ArtifactNotFoundException) at org.eclipse.aether.connector.basic.ArtifactTransportListener/transferFailed (ArtifactTransportListener.java:48).
Could not find artifact net.java:jvnet-parent:pom:3 in glassfish-repository ()
the solution was to ensure to have
:mvn/repos {"central" {:url " "}
"clojars" {:url " "}}
in the deps mapI have had this problem too. Eventually I gave up on using tools.namespace refresh. Iโm using the REPL more in the style of Stuart Halloway
You can watch this talk: https://youtu.be/Qx0-pViyIDU?t=1751 He talks about the difference in the second half of the talk if I remember correctly
sounds tough... I'm used to refresh large projects ~100 times per session. no way I could do that manually (and without errors)?
I use a lot of (comment ...)
sections to try out stuff. those also get erased when I change something and then use tools.nrepl refresh. Also I use inline def sometimes to capture stuff. All this state is erased. Something which hampers my debugging. Itโs a different trade off.
I have this โproblemโ too, I initially thought it was because of referโs, but sadly not so trivial
Hi all does anybody know if a tool that would help to refactor function calls in a clj file that were created with a :refer :all to use a prefix?
i have emacs yes. I will give it a wizz now. I don't normally use clj-refactor but have it installed
it's super sweet. especially with the Hydra menu, otherwise I'd forget all those commands
For example I have a file with (:require [korma.core :refer :all]))... and within uses (update db-table (set-fields :myfield mydata) (where (:id id)))....
(:require [korma.core :as kdb])).... (kdb/update tb-table (kdb/set-fields :myfield mydata) (where {:id id})))... etc.
The compiler will tell you about undefined symbols if you just update your import statement
just removing and relying on the compiler might be a bit deceptive as there are some function names in the korma namespace that shadow clojure.core ones - for example update and so I am concerned that I may not get compiler errors in that case
tools.deps question: when I depend on git dependency, is there a way to also specify an alias defined in that library to be used?
aliases are resolved first to determine what deps to traverse
> aliases are resolved first to determine what deps to traverse I meant to use alias defined in dependent library when specifying dependency, not using it in place where normal aliases are used
havenโt thought about it
I donโt think itโs like classifiers
classifiers select an alternate artifact
this is about selecting an alternate sub-classpath
well, feel free to file a TDEPS jira enhancement and Iโll think about it. not anything Iโm going to do soon though.
Thanks, I will! I have a use case for it, but it's not very urgent and workarounds are possible while it's not there
hi, iโm trying to process a hierarchy and i think i have things working the way i like, but i canโt seem to figure out how to avoid an atom i want to get rid ofโฆ any tips appreciated. https://gist.github.com/joefromct/0b017d43f381e3f861fc93d08e716213 I think itโs trickier than just a recur with an accumulator because there is a couple of doseqโs in the middle there.
Hereโs a version that uses reduce-kv
and reduce
instead of doseq
:
(defn recur-test [{:keys [acc val md5-list json-key]
:or {acc []
md5-list []
json-key []}
:as args}]
(let [pretend-md5 (str (hash val))]
(cond
(map? val)
(reduce-kv (fn [args' k v]
(recur-test (-> (assoc args' :val v)
(update :json-key conj k)
(update :md5-list conj pretend-md5))))
(update args :acc conj {:md5-list (conj md5-list pretend-md5)
:json-key json-key
:val val})
val)
(vector? val)
(reduce (fn [args' v]
(recur-test (assoc args' :val v)))
args
val)
:else
args)))
(:acc (recur-test {:val airplanes}))
Although note, I extracted :acc
from the final result, since the whole args
is returned from every call.
Interestingly, TDEPS-116 could also open the way for a solution to managed dependencies. The lib/deps.edn is a file that describes managed deps via aliases. app/deps.edn consumes this lib and describes which managed deps (= alias in lib/deps.edn) it wants to include as deps.
I just did a (source promise)
and discovered there's an undocumented behavior; a promise can act as a function equivalent to deliver on the promise. This is actually quite useful to me. Is this an intentional part of promise and, if so, should it be documented?
(def p (promise))
=> #'user/p
(p 37)
=> #object[clojure.core$promise$reify__8486 0x5742bdca {:status :ready, :val 37}]
@p
=> 37
@hlship for undocumented behavior you definitely shouldn't count on, I've seen deliver
used to replace (fn [f x] (f x))
which relies on that behavior
the implementation seems to quite intentionally support that
(by implementing IFn)
Iโd call it a doc bug
canโt imagine we would remove it
I would make the patch small by just extending โโฆ with deliver or by invoking the promiseโ
looking at it again, clearly deliver
is the intended entry point so I think Iโd call it an enhancement
Interesting; looking at the source for deliver, it is just (promise val)
. My thinking is that this usage in not intentional, it was just easier for Rich to re-use IFn than to define a new IDeliver interface. However, I think it is quite worth documenting and even using when appropriate.
hello all, Is there any exponentiation function of a double exponent? (function 4.554 <exponent_double>)
for both clj and cljs you can use Math/pow
via interop
user=> (Math/pow 4.554 Math/E)
61.61660979210404
You know, once you get over the parenthesis there is a lot of power to appreciate. I was about to write (defn min->ms [min] (* (* min 60) 1000)
then (defn min->ms [min] (-> min (* 60) (* 1000)))
then realized itโs just (defn min->ms [min] (* min 60 1000))
.
honestly i would absolutely hate to work in a language without parens. text editing is so convenient with them. i think a paren beginner is equivalent to a vim master
once you have something like paredit as muscle memory it seems like you really have lifted off
I am so lost in all editors that donโt sanely add parens lately
cursive spoiled me
i want to brush up on my paren editing skillsโฆ does anyone have a link to a good tutorial/docs? Iโm using doom emacs atm and smart-parens is mildly annoying at timesโฆ i think i had a better paredit setup years ago.
i use lispy and there is a file in the package that is like vimtutor https://github.com/abo-abo/lispy/blob/master/lispytutor/lispytutor.el
there might be something better out there though. Not sure if lispy is the best solution for clojure so would be interested in people's thoughts on that
Personally Iโve been using parinfer which I really love. The code practically writes itself!
In an attempt to understand datafy/nav a bit further (especially nav), I put together a very simple repo with an example here https://github.com/markbastian/datafy-playground/. The only example of significance is here: https://github.com/markbastian/datafy-playground/blob/master/src/datafy_playground/datafy_file.clj. I was wondering if some of the experts could weigh in on some questions I've had regarding these functions. Here are my questions: 1. Is it appropriate or normal to auto-datafy as you nav? In my example, if I nav from a datafied file to its parent, is there any reason to not just datafy it on the fly? 2. If a datafied item has a nested datafiable entity (in this case a datafied directory has a list of files) is it considered best practice to use collection functions to get at the item and then datafy it (e.g.
(datafy (get-in datafied-directory [:files 2]))
) or is it acceptable to nav right to it (e.g. (nav datafied-directory :files 2)
).
3. Finally, is it considered acceptable to nav to the contents of a thing if the key isn't present in the datafied coll? For example, if a file is a .csv file, is it normal to do (nav datafied-file :contents :csv)
where the contents are implied?
The example I have above can be pasted right into a REPL if anyone wants to try it out. Hopefully it will be useful in both illustrating the above items as well as being something you all can use to experiment on.Lot of good questions here, but I donโt have time or ability to answer atm and I might forget. If you put this on Clojure mailing list I will be more likely to give it the treatment it deserves
Thanks, @alexmiller & @seancorfield - And I want to make sure I don't give you guys the impression that I expect you to get to it right away. If there's a "best place" to put examples for discussion please let me know and I'm happy to put them there. I want to provide as complete examples as possible to minimize everyone else's work load.
I'm happy to continue answering in the thread (where I already posted several replies)
Hello guys, how would you make a function run one at a time, even if it is called multiple times? I have a compojure route which makes a reservation, but first it checks if there is free-space. So this is a concurrency problem, how would you solve it?
if you use an agent, they enforce a queue on all sends
@markbastian I don't know if this helps answer those questions, but the way I think about it is that you have some Thing (which is an arbitrary object -- Clojure or Java, whatever) and you explicitly call datafy
on it to get a pure Clojure data representation of that Thing. Then you can navigate that as a pure Clojure data structure using regular Clojure functions and when you want to go back to the corresponding arbitrary Thing or sub-Thing you call nav
, mimicking the pure Clojure data structure navigation. Now you're back in Thing-land, and you would explicitly call datafy
again to get to a pure Clojure data representation of that new Thing/sub-Thing.
In the case of REBL, it takes a Thing and calls datafy
to get the data it displays in the view pane and when you "drill down", it calls nav
to navigate into the Thing, and then calls datafy
to get new data to display in the view pane.
@paul931224 alternatively, compare-and-set!
or swap!
with a function that conditionally aborts
(those functions work on an atom rather than an agent though, so if the source of truth isn't in memory an agent might still make more sense)
if you can model it in an atom as a compare-and-set!, you can do the same in a sql database, and other kinds of databases often have some kind of cas
yeah, letting a db be source of truth for the transaction is smarter
I am a huge fan of a cas, so for a reservation system, cas not-reserved with reserved, and then it doesn't matter how often the function is run
@seancorfield - So, in the cases of a File or a DB thing (a row in a resultset, for example), datafy takes you from File or Row to Clojure data (likely a map). This part makes sense to me. WRT nav, are you saying nav on a map would take me back to a File, Row, or other Thing-type corresponding to where you are navigating?
Correct. Thing
-> datafy
-> data -> nav
-> Thing2
-> datafy
-> data2
The Thing you start with either needs to have metadata for the Datafiable
protocol or directly implement it.
When you call datafy
, the pure data structure produced has metadata for the Navigable
protocol (or directly implements it).
When you nav
, you get back a new Thing (with metadata for Datafiable
or an implementation of it).
Have you seen my examples in https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/datafy.clj
@markbastian nav
should not automatically datafy
things.
Did you see this thread? I was trying to answer in a thread to avoid cluttering the main channel.
In your example, you would do (-> (io/file ".") datafy :files)
to get your sequence of files, then you can either do (get X 0)
to get an individual file representation or (nav X 0 (get X 0))
to navigate to the corresponding file thing.
Pasted from main thread:
Using the "recipe" at http://corfield.org/blog/2018/12/03/datafy-nav/: You can sum this up as: Starting with a โthingโโฆ
(io/file ".")
โฆyou convert it to data (with datafy)โฆ
(-> (io/file ".")
datafy) ;Let's say this produces a map with an entry :files which is a list of java.io.File
โฆand walk it with simple Clojure data accessโฆ
(-> (io/file ".")
datafy
(get-in [:files 0])) ;This is one way
โฆand, at each stage, you can navigate to the corresponding โnew thingโ by calling navโฆ
;Option 1
(-> (io/file ".")
datafy
(get-in [:files 0])
(nav ? ?) ;Not sure how this works
;Option 2
(-> (io/file ".")
datafy
(nav :files 0) ;This could return a File or I could datafy this in my implementation
โฆwhich may return just that value or may do something more complexโฆ
โฆand from that โnew thingโ you convert it to data (with datafy) and continue the process.
Do either of these make sense? I apologize if I am sounding dense and am taking too much of you guys' time. I've watched Stuart's video, read your blog post, and the source (all multiple times) and everything seems to click except for the correct way to invoke nav.I already answered that above.
Ok, I'll take another look at nav in your navize-row function. Thanks.
Specifically this answer:
> In your example, you would do (-> (io/file ".") datafy :files)
to get your sequence of files, then you can either do (get X 0)
to get an individual file representation or (nav X 0 (get X 0))
to navigate to the corresponding file thing.
I think I may have figured it out. Please let me know if this makes sense: 1. Use datafy when you have an object (non-pure data) representation of something and want to convert it to pure data. In some instances, datafy will produce a "coordinate" that links to something else (e.g. a foreign key or hyperlink). 2. Use regular Clojure navigation (e.g. get, get-in, keywords) to navigate pure data. 3. Use nav when you need to navigate to a referenced thing. This thing should be a pointer, link, etc. It is not used to transform data. If I am getting this, places where nav would be appropriate would be foreign keys in a db, web navigation (key is :href and val is the url), or perhaps a symlink in a file system. The basic idea is there is a 1:1 link between the data and the thing you are navigating to.
I think the only thing that seems fuzzy for me is how you might handle a directory. Suppose I datafy my current directory. I could do a couple of things (This is all dependent upon my implementation of datafy):
;Option 1: Pure data
{:file "."
:files ["file0" "file1"]}
;Option 2: mixed - Pour .listFiles into a vector for :files
{:file "."
:files [#object[java.io.File 0x5cc6922c "./project.clj"] ...]}
With Option 2 you would do something along the lines of (-> (io/file ".") datafy (get-in [:files 0]) datafy) to navigate to the first file as data.
With Option 1 you would use nav. Something along the lines of (-> (io/file ".") datafy :files (nav :files 0) datafy).
Is this correct? If so, is there a preferred or canonical way of doing the implementation?Neither is correct.
๐ Sorry! Am I close?
Thing -> datafy
-> data -> nav
-> Thing2 -> datafy
-> data2.
and it's (nav <collection> k v)
where v
is nearly always (get <collection> k)
nav
mirrors the pure data "navigation" you do in Clojure
oh, whoops, Is this correct: (-> (io/file ".") datafy (nav :files 0) datafy).
I just noticed I had a double :files
Directory -> datafy
-> data structure that representations the (contents of the) directory, with metadata added for Navigable
protocol (most likely on the collection of data that represents the files) -> (get (:files dir-data) 0)
produces the first file representation, so (nav (:files dir-data) 0 (get (:files dir-data) 0))
would be the correct nav
call here -> a File (or Directory) object -> datafy
-> data structure that represents that.
(-> (io/file ".") datafy (nav :files 0) datafy)
-- no, nav
takes collection, key, value. You can't do multiple levels of navigation with it, just one.
Grokking how nav
works is definitely the hardest part of this.
It's intended to mirror (get collection key)
or (get vector index)
, and you pass it the actual value (returned by get
) as well as the collection/vector and the key/index.
From my blog post "Given an associative data representation, it does (get coll k)
first to get v
, and then it calls (nav coll k v)
to allow the underlying navigation to return an updated value."
For sure. So, looking at
(nav (:files dir-data) 0 (get (:files dir-data) 0))
I could also do:
(let[{:keys[files]} (datafy (io/file "."))]
(nav files 0 (get files 0)))
The key being that dereferencing the bare data in position 0 (coll) with the key produces the value?Which I think you just said.
But that's not (-> ... (nav :files 0) ...)
just to be clear.
Right, which is why I had to use a let to get the intermediate value.
Now I'm trying to understand why I need to provide the value.
Because nav
may just return that value, if the act of navigating is just the same as for pure Clojure data.
Specifically (let [data {:foo "bar"}] (nav data :foo "bar"))
-- nav
on a hash map is basically a no-op that just returns the value (so it's an assumption that the key you pass would refer to the value you pass).
That's assuming no metadata on that hash map.
why not just return
(get data :foo)
in the default case?In the java.jdbc.datafy
case, it ensures that the updated query
function returns something that is Datafiable
and that datafy
will add metadata for Navigable
to each row, so that when you nav
on a row, if the key matches a foreign key in the schema, instead of just returning the column's value, it fetches the related results from the DB (and starts the cycle over again).
Because it's already been passed the value (from (get data :foo)
)
Ah, plain old get doesn't allow you to add metadata.
And in the general case, some things will navigate and others will not.
In the case of a key-val being a foreign key vs. a scalar.
Yes. datafy
/`nav` are a very generic way to implement laziness.
The docstring for nav
tries to make the reason for the value clear https://github.com/clojure/clojure/blob/master/src/clj/clojure/datafy.clj#L31 but it isn't entirely obvious.
In particular, for sequences, you might only have the value, and not any matching key/index.
The default implementation of Navigable
for all Object
s is just to return the value passed in, ignoring the collection and key/index: https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/protocols.clj#L194
Ok, I think I am getting it. I suppose for a file list you might want to provide the value (file name) vs. the index since most people think in terms of a seq of names vs. a vector of filenames. Thanks!
Then you could use a filename filter in the nav protocol or something similar and not list all the files in the :files key in the datafied dir.
Yeah, the sequence/collection of files (filenames) would have metadata for clojure.core.protocol/nav
to provide how to turn the data (filename) into the actual file "thing" so the fact its a vector is less important here.
Or the metadata could close over the path to the file and so just the "filename.ext" part would be enough -- no filtering needed.
ah, nice idea
Thanks so much for all of the help on this. It's making a lot more sense.
So (nav (:files dir-data) nil "filename.ext")
would rely on the nav
implementation in the metadata of (:files dir-data)
being able to add the path to the directory in which the file lives.
yes, and that makes a lot more sense when you read it
It's tough stuff. I spent hours playing with this at Conj, trying to get java.jdbc
working with REBL after watching Stu's talk.
Yeah, I am sure being there helped a lot. Thanks again for passing on the knowledge.
Happy to help any time!
@noisesmith the problem is the source is mongodb. Read and writes can't be atomic together, I just want to put functions in queque like in core.async, but I want to wait still one ends before the other starts
agents have this behavior, but it's going to be fragile
eg. what if two servers are running your app?
@paul931224 You may need to consider here, what is going to happen if you ever load balance over more than one server ๐.
with no distributed reliable source of truth, bad things will happen
Or once you are in data-land do you capture the scope of the original thing (e.g. the FileSystem or DB) and just work in data?
you will need to stand up something like zookeeper or etcd to get cas on top of a non-transactional store
yeah, a dedicated reliable source of truth is an option
@wesley.hall I am way too amateur too consider this yet. This is my third full-stack application till now, and this one have real clients, so this is my first concurrency problem ๐
or a db that provides this while also storing your data - such things exist
well I just want to block other operations while I read, calculate then update the new value
so a queque with blocking ๐
@paul931224 Well, here's an interesting option. If you choose to represent each "ticket" in your mongo database as an individual document, you should be able to use the mongo function, "findOneAndUpdate()", to find a "free ticket" and reserve it.
@wesley.hall well this is a bit more complex, I store free-times in minute intervals [[600 1000][1200 1440]]
... so I can't update with mongo logic, I need my clojure power to make new ranges of minutes etc.
my all reservation logic is about ranges, remove ranges from ranges, then keep the new ranges
@paul931224 Honestly, you might need to rethink that if you want a robust solution to this problem. It's not uncommon to have to do this, particularly with something like mongo. You often have to design your data storage around operational requirements.
You could also look at java.util.concurrent if nothing obvious to be found in clojure. Lots of "locky" type stuff in there, but locking your entire system while one person makes a booking strikes me as a... non-ideal solution.
Incidentally, many large systems solve this problem by just "overselling" and solving the problem later ๐.
I want to lock only the function. only adding reservation and modifying it, is a problem, which happens like 40 times a day, the problem is it happened at the same time once, so reservation overlapped each other.
Moved to separate thread...
well, I thought it wont be a problem for a time myself
but in first week it happened 2 times ๐
it is like 2 guys picking same date, same hour, at same guy, in the same 5 ms-es
well, anyway, I need an easy fix, because I can't make a new architecture in one night. For example storing a state in atom, setting it to true if running, false after the update, and the function should recur still it is false, so it can run and set it true and etc...
@markbastian I answered in a thread to avoid cluttering the main channel.
@paul931224 Maybe look at (locking)
, you can just def some kind of object in the namespace and lock on it. I am not sure I would advise this as an actual long term solution, but it will probably do what you need for now.
btw my first suggestion, using an agent, is effectively a queue plus locking plus execution on a separate thread
looking in both, your help is much appreaciated ๐
locking might be better, making things async via threads might just increase complexity here
Yeah, in retrospect, not even sure how you do this with an agent, more asynchronity there, not less. Though in fairness I don't use them much, so might be missing something.
you send
the reservation setting function to the data defining reservations
the setting functions are guaranteed to not run concurrently
they run async from the caller, but no two agent functions on the same agent overlap
so agents are perfect for a case where you need a single source of truth, no retries, no overlapping changes as long as it's OK for all of this to be async from the perspective of the code doing the modification
well my customer will mention it for sure ๐
Well, that does seem to be the airline model. Over sell and kick the can down the road to the poor customer service person at the airport.
yeah, a given agent will wait for the result of its last execution before starting its next.
Definitely took me a while to re-grok that fact. References to, "thread pools" in the (send)
docs are actually misleading because it does suggest that sends are done concurrently, but what is really meant here is that there is a single thread pool that is shared across all agents for (send)
. I think I did know this once... but honestly, how often do you use this stuff? ๐