This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-01-08
Channels
- # alda (1)
- # announcements (18)
- # babashka (101)
- # beginners (110)
- # calva (17)
- # cider (53)
- # clara (18)
- # clj-kondo (26)
- # cljdoc (6)
- # clojure (152)
- # clojure-europe (9)
- # clojure-portugal (4)
- # clojure-spec (20)
- # clojure-survey (7)
- # clojure-sweden (10)
- # clojure-uk (10)
- # clojured (1)
- # clojurescript (29)
- # core-async (7)
- # cursive (4)
- # datomic (11)
- # defnpodcast (2)
- # dirac (1)
- # emacs (13)
- # events (2)
- # figwheel-main (1)
- # fulcro (1)
- # jobs (14)
- # jobs-discuss (17)
- # leiningen (2)
- # malli (1)
- # off-topic (74)
- # overtone (1)
- # pedestal (4)
- # planck (2)
- # re-frame (7)
- # reitit (4)
- # remote-jobs (4)
- # shadow-cljs (78)
- # slack-help (3)
- # spacemacs (56)
- # test-check (3)
- # tools-deps (6)
I'm having some difficulties grasping Manifold streams & Aleph networking. As I understand, you have sinks and sources, which basically means "something-to-put-into" and "something-to-get-data-from". Say I want two different clojure program to communicate via a TCP port (many clients <-> one server). If I use manifold streams and aleph tcp, do I need to take care of: 1. Ordering of data sent/received? 2. Integrity of data sent/received? I get that a lot of stuffs has been abstracted away so I just need to have a stream from the source (client) then I send it into an aleph tcp-client, then on the server I (may or may not?) get the data that was sent. What I don't get is: do I have to take care of ensuring if data was sent fully and in the right order (1&2), what if in the middle of a client sending data, another sends in data and the server gets mixed up between the two (1&2).
In the examples, Gloss is used to ensure a protocol is agreed between clients and servers (e.g. what comes first and what comes last). But if data is fragmented into tcp packets, do I need to ensure protocol is neatly divided into tcp packets, or does it get taken care of already?
finally, what/how can one google/learn from documentation about what a certain java stream objects can be converted into in clojure? This is where I usually stumble because of juggling between Java Documentation and (pure?) Clojure data definition --- what's the right way to look into a returned data and know what it is, if it's java or not, ... from a repl point of view ?
I don't know much about Gloss, manifold, or aleph, but I do know that TCP is a byte stream service, so you should never need to worry about how bytes are broken up into packets, or about the ordering of bytes within a single TCP connection (they are guaranteed to be delivered to the receiving application in the same order they were sent). If any of these libraries handle automatically reconnecting a TCP connection that becomes broken, there might be ordering issues between different TCP connections that are yours to deal with, i.e. I know that TCP has no ordering guarantees between different connections.
Thanks andy. I infer from what you mean about "single TCP connection" meaning (by design of TCP) two different TCP connections would (have to) be distinguished if any tcp library is to be taken seriously (i.e. not aleph but if X purports to be a TCP library, it should by definition handle what you said and this)
Some application level messaging libraries try to "hide" some details of when TCP connections are established, broken, and perhaps automatically reconnected for you, under the layer of the API. If so, whether there are ordering guarantees across connections breaking and being reconnected is something you would need to find out about from that library's documentation. TCP by itself cannot do anything to guarantee that.
alright. I understand what you mean. 😀 it's the asynchronous tcp that makes my head hurts a bit. I'll read more into the library itself
I find this discussion on if/when/how to use comp
being very enlightening: https://clojureverse.org/t/to-comp-or-not-to-comp/ I also find that I can read it several times and find new pieces to the, yet tiny, Clojure and functional puzzle that is slowly assembling in my head.
Has anyone used clojure to verify if an email address is valid and exists on a server?
The only sure way is to send it email and receive data via a new http request - e.g. a verification link. it's also the simplest and most secure.
Weavejesters library of validator predicates does something funky that sounds like what you want.
It doesn't guarantee who the owner is, but I think it gives you a good idea that it exists.
yeah, I don’t want a lot of info about the email or the person behind it, just “does it exist on the server?”
I’ll look into it, thanks @U09LZR36F, also @U0509NKGK for chiming in 🙂
yes, the library has a predicate valid-email-domain?
that takes a step in the direction I need, nice 😄
there are also services like clearbit or http://kickbox.io
I'm confused about what kickbox is actually selling. One could interpret their marketing to suggest that they will email everyone in my mailing list and ask if they are real. I'm sure they aren't, but still.
@U09LZR36F you can use their free API to verify if the email address is valid, disposable (mailinator etc) or coming from a free account (gmail, yahoo). If you just want to know that the email address can receive messages - then your only reliable solution is the verification link
I am experiencing a problem with with-redefs
in my clojure test.
Here is my setup
namespace a -> requires namespace b and has a function foo
namespace b -> requires namespace c
namespace c -> has a function bar
Test namespace a-test
I want to test function foo
but I want to redef bar
Here is my implementation
(ns app.a-test
(:require [app.a :as a]))
(deftest foo-test
(testing "retuns something"
(with-redefs [a.b.c/bar (constantly true)]
(is (= true (foo "test"))))))
However, here still the actual bar
is called! What am I missing here. Please adviseI wasn't even aware that a.b.c was valid way to access a ns.. Why not just require the c namespace directly e.g (:require [app.c :as c])
I'm not sure thats the issue, as i'm also not familuare with with-redefs
.
(with-redefs [a.b/bar (constantly true)] (+ 1 1))
throws when a.b/bar can't be resolved
The problem is even requiring c
won’t work here. I tried that as well. It still calls the bar
method.
I'm not that familiar with with-redefs
but it feels like the issue can be in the app.a
namespace. Can you give the code where foo
is defined and bar
is used in there?
(ns app.a
(:require [app.b :as b]))
(defn foo [user provider]
(let [params {:user user :provider provider}]
(b/sample params)))
(ns app.b
(:require [vault.core :as c]))
...
...
(defn sample [params]
(c/read-secret (client) params))
@U11BV7MTK There are no async functions. However there is a defmethod in it. Not sure if that causes any problem
when you say "bar method" do you mean that bar is defined via defmethod?
Yes, you are right @U051SS2EU
Interested in it myself. I can't find why that wouldn't work - defmulti
uses defonce
internally, which still uses def
.
minimal repro that what @USFNA0VPH wants does work - I suspect there's something else going on
Clojure 1.10.1
(cmd)user=> (ns c)
nil
(cmd)c=> (defmulti bar :type)
#'c/bar
(cmd)c=> (ns b (:require c))
nil
(cmd)b=> (defmethod c/bar :x [m] (:y m))
#object[clojure.lang.MultiFn 0x5b43fbf6 "clojure.lang.MultiFn@5b43fbf6"]
(cmd)b=> (defn foo [x] [(c/bar x)])
#'b/foo
(cmd)b=> (ns a (:require [b] [c]))
nil
(cmd)a=> (with-redefs [c/bar (constantly true)] (b/foo {:type :x :y 42}))
[true]
I scratched my head a lot to understand what’s going on here. Couldn’t figure out why It was not working. Finally ended up using the mock client from vault for testing instead of with-redefs
Anyone using https://github.com/ptaoussanis/sente ever have the desire to make a request on the client with a callback that expects multiple responses? ie to call ?reply-fn multiple times on the server? This is not currently supported (send messages with callbacks only support a single reply). I realize I can replicate this by implementing my own callback id machinery, but I sort of wish I could piggyback of sente’s internal calback ids. Is there a good way to deal with this that Im missing?
I have something like that - for some operations, I need to report progress. And yeah, I've done pretty much what you describe, and the server calls a special RPC called :on-progress
or something like that.
Good to know. I suppose the main reason not to have this functionality offered directly by sente is that it would then need to expose a means to unregister a callback manually and it doesn't want to support that additional complexity?
You should ask Peter Taoussanis, not me. :) Maybe it's something that could be implemented in a way that would benefit all, I don't know. But right now it seems that there can be too many corner cases where each project would require something special.
I'm working with Redis in my application, and I'd like to have an in-process mock of my Redis DB for development and testing purposes. Right now I have a protocol with my common DB access fns, and two places I reify
the protocol: one with the real Redis impl and one with a clojure map in an atom as my "db". I'd like to be able to easily inspect the whole map, but since it's part of the reify
closure I'm not sure how to easily get at it. Is there a better way to do this?
I thought about using defrecord
but since the fields are immutable I think that excludes using my map in a mutable atom?
you can store an atom inside a field in a defrecord, and then implement the protocols via updating / accessing that field
I had exactly the same problem in the past - and it was not feasible to create a protocol which implements all possible redis functions. So I wrote a Redis component, which works a bit more like the redis CLI/protocol: https://github.com/nomnom-insights/nomnom.omega-red
it's slightly brittle as you'd need a wrapper around the default constructors to ensure an atom is created, but after that it would just work
With it creating a mock is pretty simple: you create a record which implements the Omega Red protocol and store the state in an atom
Omega Red looks a little heavy for what I'm looking for at the immediate moment but I'll definitely check it out for future work, thanks!
@U051SS2EU I control enough of the calling code that I can just pass in (atom {})
to the constructor each time if I need to. Will that work?
Sure thing, I meant it more as a suggested approach - we use Redis quite heavily in different forms so it made sense to create a Component for it
that works too, yeah - simple demo coming up
(defrecord MockRedis [db] IRedis (blah...
Later... (MockRedis. (atom {}))
(ins)user=> (defprotocol Db (store [this k v]) (access [this k]))
Db
(ins)user=> (defrecord StubDb [storage] Db (store [this k v] (swap! storage assoc k v) v) (access [this k] (get @storage k)))
user.StubDb
(ins)user=> (def db (->StubDb (atom {})))
#'user/db
(ins)user=> (access db :some-key)
nil
(ins)user=> (store db :some-key 42)
42
(cmd)user=> (access db :some-key)
42
@UANMXF34G just fyi it's pretty much always better to use the auto-defined constructor ->Foo
rather than the interop constructor Foo.
(but they do end up doing the same thing)
Any specific gotchas when using Foo.
or is it just a style thing?
the function version is smarter about reflection, and less likely to hit some gotchas when using the record from another ns
hmm, okay good to know, thanks
to use Foo.
you need to both require
and import
, ->Foo
only needs require
Yeah based on my reading so far I figured I'd have to tackle the import
issue at some point so it would be nice to avoid needing that.
also the auto-generated map->Foo
is great when you have more than ~2 fields defined
ooh, cool
Is there a trick to getting this working when my protocol is defined in a different namespace than the defrecord
is defined in?
I get Unable to resolve symbol for the protocol method name when I call the method on a record instance.
the protocol method belongs to the ns that defines the protocol
so use that ns, not the record's ns, when invoking
gotcha, I think that's working
clj -A:graph
is rad! Very helpful in reasoning about dep trees.
On a related note, is it possible to get the clojure
/`clj` cli tools to list 'dependency differences' and 'dependency conflicts'?
• Some libs we're using are in disagreement about the best version of Jackson.
What's -A:graph
? :graph
is just an alias somewhere in one of your deps.edn
. What does it specify?
Maybe walmartlabs/vizdeps
it can specifically only show conflicts and the origin
@U0E2P47B7 😉 - that's the functionality I'm trying to recover in deps.edn land.
I've been thinking about it. There are some challenges. The trace mode stuff for tdg does actually show some of this but it's not a very efficient way to consume that info.
I'd love to be able to produce reports like this one automatically: https://github.com/aws/aws-sdk-java/issues/1108
@hhausman -Strace
will help you there.
It produces a trace.edn
file listing every dependency clojure
looked at, which one it chose, and why.
The raw info is in there but I've been thinking about ways to improve it. It's on my list of things to look at the next time I take a cycle through this stuff
@seancorfield - that is nice, thank you.
@alexmiller - for sure, super-grateful for your efforts so far. I can imagine tools that consume trace.edn
and produce actionable insights.
if you do something like clj -A:graph -t -o trace
you will get a series of numbered images trace100.png, ...
if you step through those, that's kind of a visual depiction of what it's trying at each step, what it chose, etc
I've been thinking about how to collapse all of that info into one graph
they might end up looking like that graph you posted in the thread
The little red lines that vizdeps
had were appreciated in this regard.
The classic "Possibly confusing dependencies found:" output from lein deps :tree
was never as user-friendly as I'd like, but was another approach.
make sure you try the new --size option too :)
Is there a benchmarking analogue to clojure.test
? Perhaps one that makes use of Criterium? I'm looking to define suites of benchmarks, generate reports, etc.
This feels like a long shot, but I'd love clojure.tools.logging
not to print the entire namespaced keyword. Like printing :f.b.b.q/customer
instead of :foo.bar.baz.quux/customer
. Now that I'm using spec, my logs are really dense with namespaces I don't really care about
You can change the data that your code sends to clojure.tools.logging calls...
alternately, make your own locally modified version of clojure.tools.logging that makes replacements in data sent to it, but changing what you send to it seems better in the sense that the replacements you want are specific to your application, it seems.
I just want to pass a map to a logging call, without having to transform it into a new map with different (un-namespaced) keys
Would love to know what that flag is, I couldn't find it on Google 🙂
Default behavior is this in Clojure 1.10.x:
user=> {:foo/bar 1 :foo/baz 2}
#:foo{:bar 1, :baz 2}
But it only prints that way if every key in the map has the same qualifier/namespace.
Whether you pass a modified map, or something changes the way the map is printed after you make the logging call, someone has to write that code, which I don't believe exists today. What if you have two namespaces/qualifiers that would become identical after abbreviation?
Indeed, it's not a perfect solution. It's really just to reduce log verbosity. It can be hard for a human to parse it visually when there's a lot of "meaningless" cruft in there.
@U0CMVHBL2 The repl binds it to true, but otherwise it defaults to false.
This is excellent. I think I see the problem though - if you have keys from different namespaces in the map, it falls back to printing the full namespace: Desired
user=> {:a.b.c/bar 1 :a.b.d/baz 2}
#:a.b{:c/bar 1, :d/baz 2}
Actual
user=> {:a.b.c/bar 1 :a.b.d/baz 2}
{:a.b.c/bar 1, :a.b.d/baz 2}
Oh well ¯\(ツ)/¯That is a feature, not a problem, for the people that wrote that code 🙂
They wanted something that could be read back in and know it was equal to what was printed. They did not have your use case in mind, because your use case loses information.
I'm not making any normative statements here
I was just curious if anything out there existed that matched my individual requirements. No big deal if not
(defmethod print-method clojure.lang.IPersistentMap [v ^http://java.io.Writer w] (.write w "{") (reduce-kv (fn [_ k v] (.write w (str (if (qualified-keyword? k) (str ":" (name k) " " v) (str k " " v)))) (.write w ", ")) {} v) (.write w "}"))
Otherwise your best bet is to wrap the logging functions or have a util fn that strips out the qualifiers that you call on the maps before passing them to tools.logging
Is there a trick to getting this working when my protocol is defined in a different namespace than the defrecord
is defined in?
easy? one for you quys this time. I'm in the middle of clojurifying a big, rather serious, enterprisy java software. clj conventions seems to be not to use fully qualified package names (i.e. com.mycompany....). I can (re)name packages to whatever I want before I give it for wider consumption but afterwards, it will be a challenge. is there a reason not to use fully qualified package names?
they're not as popular in the clj community, but that's an anti-pattern not a pattern. it's been mentioned a few times that generally we should
yeah, that's kind of my reading on it, thanks. anybody else with other data points why I shouldn't keep em?
I think it's fundamentally a conflict between common lisp aesthetics vs. java pragmatics
a good compromise is to use a unique enough project name that it won't conflict, and otherwise company (if unique enough) or com.company
related: foo.core
is a weird convention, and arguably you should never have com.company.foo.core
(and many people would even suggest avoiding foo.core
and instead having a more meaningful multi-part namespace like company.foo
)
The thing with core
is that it can mean anything, which can lead to conflated namespaces
if you split it in api
, main
etc, things can be more self-evident for consumers and maintainers alike
i have an error from AWS cloudwatch which is a string containing EDN. All the strings inside this EDN data structure are invalid clojure strings because they are backslash escaped. I'm wondering if anyone has a clever solution how to parse this as edn in order to pretty print it. Right now I'm just using some shell scripts to do it
Could it be that you can call read-string
on it and it would give back Clojure data?
Shoot, never mind. If the "outer quotes" have backslashes in the text you have, read-string
will not do it.
Question on tagged literals and metadata: Some tagged forms will have line/col metadata available, but anything that isn't IMeta
will not. Is there any other way of accessing that info in the context of a data reader?
Clojure's built-in reader and the contrib library tools.reader have that limitation, I believe.
There are other libraries for reading Clojure data/code (perhaps with limitations of what they support -- not sure about tagged literal support) that return custom data structures that are not just the Clojure data alone. I don't recall all of them right now, but one is https://github.com/cgrand/sjacket
I have not used one for any serious use case to give feedback comparing any of them.
Interesting, thanks.
If you call read-string
on that one time, you get back a Clojure string. If you call read-string
on that string, it would give you a Clojure map.
user=> (clojure.edn/read-string "{:msg \"error response triggered\"}")
{:msg "error response triggered"}
user=>
I think the argument to read-string
that you showed in your sample was in a file, not already in the code in question.
but I could be misinterpreting the situation.
https://gist.github.com/nxtk/7c646697240fff2833d104211c8f36c1
guys would you be kind to review my lazy iterative hanoi tower implementation, especially if i can do something about if odd-n?
branch to make it more compact\readable
algorithm explanation https://cs.stackexchange.com/a/96669
using letfn just to make helpers private is not really idiomatic, for starters