This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-02-01
Channels
- # beginners (117)
- # boot (8)
- # cider (22)
- # clara (3)
- # clojure (79)
- # clojure-italy (3)
- # clojure-spec (34)
- # clojure-uk (34)
- # clojurescript (74)
- # cursive (7)
- # datascript (5)
- # datomic (28)
- # defnpodcast (1)
- # dirac (9)
- # docker (25)
- # duct (1)
- # emacs (14)
- # fulcro (67)
- # graphql (1)
- # hoplon (15)
- # jobs (4)
- # juxt (6)
- # off-topic (76)
- # parinfer (3)
- # re-frame (25)
- # reagent (4)
- # rum (6)
- # shadow-cljs (1)
- # spacemacs (30)
- # sql (15)
- # unrepl (36)
- # yada (1)
What makes it "a clojure 1.9 library"?
If it uses features of Clojure 1.9 that were not in Clojure 1.8, then it might be a bit tricky.
Via Boot pods maybe?
Hi guys, so I'm trying to create a web service for work. Basically what this service does is that it processes request from external clients and when doing so, it communicates with other services. Some request can have as much as 5 services to communicate with in order to be processed.
There are things I'd like to get advice for, since I'm new to clojure and don't have "production grade" experience with it.
How would you guys reuse HTTP connection? clj-http
has a connection manager, but I'm not sure on how to use it properly, since it state that it needs to be shut down after use. Is the resource (I don't know, maybe socket or file descriptor) is forever going to be unrecycled even if the java jar process has stopped? I apologize if this sounds trivial, as my understanding of Java is pretty shallow.
no resource is unrecycled outside the jar process - it uses a thread pool, and an explicit shutdown of java will close that pool. IIRC it will also be shut down if you explicitly call (shutdown-agents)
but it should be easy to find out for yourself
there's no mainstream non-broken OS where you can have a process fully exit and deny the usage of a socket to other processes
Alright, thanks @noisesmith, I was afraid I need to setup some mechanism to gracefully close all resources when the process is signaled to stop.
@bronsa @tbaldridge: (re discussion weeks ago on jvm / bytecode) Here's a minimal basic "dsl -> bytecode" supporting 'variable names' instead of [:iload const]
(do "* gen-func"
(defn vks->kv [m]
(into {} (for [[v ks] m, k ks]
[k v])))
(def jvm-field-size (vks->kv {2 [:long]
1 [:int :float :farr :iarr]}))
(do "** vars->locs"
(defn vars->locs [vars]
(second (reduce (fn [[offset mp] [nm kw]]
[(+ offset (get jvm-field-size kw))
(assoc mp nm [kw offset])])
[1 {}] vars)))
(vars->locs '[[a :long] [b :int] [c :float] [d :long] [e :farr]]))
(defn cexpr->bc [vars]
(fn me [expr]
(cond
(number? expr) [:ldc2 expr]
(symbol? expr) (let [[kw offset] (get vars expr)]
[(case kw
:long :lload)
offset])
(seq? expr) (case (first expr)
do (vec (map me (rest expr)))
l+ [(vec (map me (rest expr))) [:ladd]]
l* [(vec (map me (rest expr))) [:lmul]]
lreturn [:lreturn]
))))
(defn cfn->bc [[nm inputs ret locals & body]]
{:name (str nm)
:desc (conj (vec (map second inputs)) ret)
:emit ((cexpr->bc (vars->locs (concat inputs locals)))
(cons 'do body))})
(defn lst->bc [lst]
{:name "my.dyn.TestClass"
:fields []
:methods (mapv cfn->bc lst)}
)
(defn gen-obj [lst]
(let [co (ic/visit (lst->bc lst))
obj (ic/new-instance co)]
obj))
(def code '[(test0 [(a :long) (b :long)] :long
[]
(l+ (l* 2 a) b)
(lreturn))])
(.test0 (gen-obj code) 2 3)
)
is there yet a clojure reader (tools.reader?) that can read cljc conditionals unevaluated?
someone helped me find the option flag sometime ago, but unfortunately, I can't find it right now
okay cool thanks
Why would a function like this throw
Caused by java.lang.IllegalStateException
Attempting to call unbound fn:
#'my-project.utils/build-event-stream
(defn build-event-stream
[{:keys [builder input-topic output-topic check-fn prepare-fn schema schema-registry]}]
(->
(.stream builder input-topic)
(kf/map-values avro->clj)
(kf/group-by-key)
(kf/aggregate check-fn
{}
(Materialized/with (Serdes/String) (EdnSerde.)))
(kf/to-stream)
(kf/filter event?)
(kf/map-values prepare-fn)
(kf/map-values #(map->avro schema %))
(kf/peek #(println %1 " - " %2))
(.to output-topic (Produced/with (Serdes/String) (build-avro-serde
schema-registry)))))
is it trying to call build-event-stream
somewhere that's not visible here? look at the stack trace to see where it was calling from
I'm calling like this
(u/build-event-stream {:builder builder-frequency-events
:input-topic new-electric-data-topic
:output-topic frequency-events-topic
:check-fn check-frequency-event
:prepare-fn (u/build-prepare-function
[:read-at
:customer
:freq-t
:freq-a
:freq-b
:freq-c]
[:freq])
:schema sch/frequency-event-schema
:schema-registry schema-registry-url})
All of these are map values are well defined
Even when I call like this
(u/build-event-stream {:a 3})
It gives me the same exception
If I load the build-event-stream namespace buffer manually with cider, it works, but in a new restart, it doesn't
@gfredericks Working now, was related to some imports... java at the way, thanks!
I have a program that fetches some data from an SQL server and does some transformation on it. What’s the best way of testing both the querying and the transformation?
Currently I have some clojure.test
tests and a fixture like (use-fixtures :once fetch-data-and-store-in-an-atom)
and some deftest
s that use the atom. I’d then execute it with something like DB_PASSWORD=1234 lein test
.
unit tests aren’t unit tests if they rely on an external system; consider mocking out the database calls and purely testing your code without involving a db
But ignore that if you are past that decision and really want to know how to involve the db
assuming there's a function that fetches the data from the database, mock it with with-redefs
and have it return data that resembles real data
if you want to access the db that's not an unit test but it is an integration test, so...valuable but not the same
we use postgres in production, and for our unit tests we make use of https://github.com/opentable/otj-pg-embedded which downloads & runs a real postgres server on the machine running the tests. Just have to set up the tables in the initialization. Don't know if something similar might exist for SQL server?
FWIW, we use Docker locally to run MySQL, Redis, MongoDB (and a custom search engine) to support our test suites -- with DB migration scripts to tear down / set up test data. But @alex438 is right that it's not really a "unit test" if you're involving a DB in your test.
In some of our tests, we have a protocol covering the storage operations that are required and then we mock the whole thing for tests. That works for a specific API that is implemented underneath as a set of queries and updates, but doesn't support arbitrary queries. And you potentially still need some level of integration testing or UAT to ensure it all works with a real database.
@alex438 Yes, they definitely are not just unit tests. They are supposed to test data transformations on the real data.
What about downloading a sample of the real data and storing it in a test resources directory in your project, that your unit tests could then load when mocking out the db connection?
@alex438 Yeah, so have a separate process fetch the data and then run the unit tests. Sounds like a good idea.
@seancorfield I’m seeing some funkiness with boot-tools-deps and Metosin’s alt-test. I’m digging in a bit, but before I get too far in, are you aware of any issues?
@grzm Can you be a little bit more specific please? 🙂
(in particular, boot-tools-deps
assumes you are using deps.edn
files instead of setting :dependencies
in build.boot
, and that you are using aliases for management of dependencies and classpath stuff, rather than leaning too heavily on :scope
in dependencies)
@seancorfield I’m seeing errors like “java.lang.RuntimeException: Unable to resolve symbol: deftest in this context”,
In other words, the use case for boot-tools-deps
(that I need to explain better in the README!) is that you have a project that "runs" fine with clj
first -- and you use boot-tools-deps
to add more Boot functionality on top of that.
One of the things I liked about boot-tools-deps was that I could keep boot-related dependencies in build.boot
and out of deps.edn. That doesn’t seem to be the case anymore.
@grzm You'll need to share a lot more detail about your setup and exactly what you're trying to do... Feel free to do that in either #tools-deps or #boot depending on which part you think you're having problems with.
That would be perfect, thank you!
how do I get one deftest to call another deftest, i.e. I'm trying to setu pa hierachy of tests, and have one deftest call all the 'smaller' tessts
(ct/deftest test-jvm-misc
;; ireturn
(ct/is (= 42 (.go (rbc [:int] [[:ldc 42] [:ireturn]]))))
;; iload
(ct/is (= 2 (.go (rbc [:int :int] [[:iload 1] [:ireturn]]) 2)))
;; goto
(ct/is (= 20 (.go (rbc [:int] [[:ldc 20] [:goto :END] [:ldc 30] [:iadd]
[:mark :END] [:ireturn]])))))
(ct/deftest test-jvm-arith
(ct/is (= 50 (run-arith :iadd 30 20)))
(ct/is (= 10 (run-arith :isub 30 20)))
(ct/is (= 600 (run-arith :imul 30 20)))
(ct/is (= 1 (run-arith :idiv 39 20)))
(ct/is (= 19 (run-arith :irem 39 20))))
(ct/deftest test-jvm
;; how do I call the other two tests
;; i.e. test-jvm-misc and test-jvm-arith
)
@qqq deftest
just generates a function with the name you give it, so you can just call (test-jvm-arith)
and it will work.
However, you probably don't want to do this in a test suite, since clojure test runner looks for functions with the testing metadata and runs them, so when you run all the tests, you would run test-jvm-misc
, test-jvm-arith
, and then run both of these again when test-jvm
is run again.
The is
macro doesn't have to be inside a deftest
, so you might consider just making test-jvm-misc
& test-jvm-arith
normal functions, which are called inside test-jvm
, so the tests only get run once but are still organized in a way you're happy with?
okay, so the answer is two parts: 1. the solution is
(deftest test-jvm
(test-jvm-misc)
(test-jvm-arith))
2. I don't want to do the above since (test-jvm-misc) will be called twice (atleast)Yep! But like I said in the second response, you can just make test-jvm-arith
& test-jvm-misc
normal functions, then they'll only be tested once when test-jvm
is run with the test runner
okay, so the 'solution' is:
(def test-jvm-arith
(ct/is ...)
(ct/is ...))
(def test-jvm-misc
(ct/is ...)
(ct/is ...))
(deftest test-jvm
(test-jvm-arith)
(test-jvm-misc))
@qqq instead of def test-jvm-arith
do defn test-jvm-arith
(and same for test-jvm-misc
)
of you do it as def
, it will run the is
macro when you compile the code 🙂
I'm using leiningen version 2.5.3 and I originally installed it using brew. One of my projects is asking me to upgrade to version 2.7.1. I tried 'lein upgrade' but it gave me an error:
lein upgrade
This version of Leiningen was installed with a package manager, but
the upgrade task is intended for manual installs. Please use your
package manager's built-in upgrade facilities instead.
I then tried to run brew install leiningen but it responded with:
Updating Homebrew...
==> Homebrew has enabled anonymous aggregate user behaviour analytics.
Read the analytics documentation (and how to opt-out) here:
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
Error: Failure while executing: git config --local --replace-all homebrew.analyticsmessage true
Error: leiningen 2.5.3 is already installed
To upgrade to 2.8.1, run `brew upgrade leiningen`
lein is a shell script on your path, plus a cache under $HOME/.lein/
you can tell lein to change versions without deleting or uninstalling it with lein upgrade
or lein upgrade specific.version
(can be older or newer than current, it doesn't care)
@tlomberg brew uninstall leiningen
should delete the existing one installed through brew; then install the lein shell script as specified on https://github.com/technomancy/leiningen#installation (and then you can follow the directions that @noisesmith gave you)
@tanzoniteblack what I was trying to get at was that even if brew was totally broken and unfixable, it's sufficient to either delete a directory plus a file, or tell leiningen to change versions - but yeah definitely better to use brew as intended if using brew
@tanzoniteblack @tlomberg
re-reading, you don't need to uninstall the brew leiningen to get 2.7.1, it suffices to install 2.8.1 and then run lein upgrade 2.7.1