Fork me on GitHub
#clojurescript
<
2017-05-06
>
lsenta05:05:31

Hi #clojurescript what's the correct way to have private API keys as env variable, that can be imported in a env.cljs? As far as I understand, that's a little peculiar since the evaluation should happen at compile time

danielcompton05:05:50

@lsenta I have a half finished blog post about this

danielcompton05:05:57

gotta go to dinner

darwin06:05:06

one reason is that you have to use type hints to make DCE work properly with closure-defines: https://github.com/binaryage/cljs-devtools/releases/tag/v0.5.3

darwin06:05:32

(if ^boolean goog/DEBUG (devtools/install!))

darwin06:05:49

without ^boolean it won’t properly DCE it

rauh06:05:22

Yeah, I've been using a (onlydev (foo) (install!)) macro for years. Easier than the constant type hinting IMO.

martinklepsch06:05:31

@danielcompton you can provide them to :closure-defines from the environment, no? :closure-defines {:app.core/production (System/getenv "PROD")}… lets you avoid some macros at least

darwin06:05:29

I tend to use macros, which are more predictable IMO. If I decide not to emit some code, I’m 100% sure it won’t appear in :advanced build.

darwin06:05:59

but both solutions are fine, just with :closure-defines one has to be more careful

darwin06:05:40

e.g. this was another issue (not really a problem of :closure-defines, but related): https://github.com/binaryage/cljs-devtools/commit/4e368408986bf739b56aad64e064b854fd7989d2

martinklepsch06:05:27

yeah, in the light of stuff like this I think I understand why you favor macros

noonian06:05:43

I like the macro approach because, if you understand macros, you can read the source and understand exactly what is happening. If you are a newcomer and haven’t fully grokked macros yet the opposite may be true.

darwin06:05:28

these days I tend to merge: 1) some kind of default config, 2) :external-config passed into cljs compiler options 3) and env config, at least for library code

darwin06:05:45

and emit result into cljs as data

darwin06:05:29

not that one can implement nice validation on top using clojure.spec 🙂

lsenta06:05:42

Hum, thanks for all the links, I'm still trying to wrap my head around how you get your env variable into the clojurescript code

lsenta06:05:17

It seem's to me that all your example shows code that relies on having the variable already in your project.clj

lsenta06:05:36

Ok, you rely on the macro step being called from java

lsenta06:05:48

which has access to the env as any jvm process

darwin06:05:56

here is the macro:

darwin06:05:19

that '~ is important, to treat everything as data

darwin06:05:01

ignore the delay call, that is there to allow DCE

darwin06:05:41

for curious, here is the explanation why it was needed, see point #1 here: https://github.com/binaryage/cljs-devtools/issues/37#issuecomment-293575471

lsenta06:05:13

Could I get away with something as simple as

(defmacro get-env [name & [default]]
  (let [val (or (System/getenv name) default)]
    `'~val))
?

noonian06:05:22

Almost, you don’t need either of the quotes on val since you will just return it as a literal

noonian06:05:53

nor the unquote

noonian06:05:23

(defmacro get-env [name & [default]]
  (or (System/getenv name) default))

lsenta06:05:50

Haha I've been overthinking the symbol transformation, thanks @noonian

noonian06:05:01

No problem

lsenta06:05:38

Thanks for the details & answers, I'll stick with the straightforward solution for now

lsenta06:05:07

Glad to know you guys have badass solutions but secretly hoping to never need it 🙂

cmal06:05:55

Hi, are there any examples that uses cljs/re-frame + express.js?

lsenta07:05:24

Hey @cmal what do you intend to do? For example, I use re-frame to build an SPA, which means that my backend only serve the files & provide a storage API. express.js would serve a REST API in my case.

cmal07:05:18

Do you have a demo repo? I'm afraid I cannot express my work very well, I want to first dive into your demo repo and have a further description of what I need for it. Thank you!

cmal07:05:41

I do the WECHAT authorization process in the server side and want to use express.js to serve my static files and serve the template html files. I also want to have the template can receive some parameters from server side.

lsenta07:05:41

Good so both are independant

lsenta07:05:25

If I where you I'd follow one of the hundreds tutorials on express to setup my backend, you'll probably write a small REST API that a client can send request to

lsenta07:05:51

Then look into reframe and cljs-http this template should be a good start: https://github.com/Day8/re-frame-template

cmal07:05:54

OK, Thank you. I use re-frame-template but I do not know how to let the app.js generated by re-frame-template project serve as a template for express.js to use.

lsenta07:05:53

You're going to use templating to pass data to your user

lsenta07:05:02

use express.js to template your index.html

lsenta07:05:08

and in your index.html generate some stuff like <script>my_app_value = {{ templated_string_xxxx}}</script>

lsenta07:05:07

when you're clojurescript code loads, use the data from there, in a var like js/my_app_value. Don't compile with advanced optimizations.

lsenta07:05:46

That's a hacky solution, but it's simple and you'll come back later to fix it.

thheller07:05:53

@lsenta you'd just need externs for /** @const */ var my_app_value; to keep advanced optimizations

thheller07:05:21

or access via (js/goog.object.get js/window "my_app_value")

lsenta07:05:20

Or use oops

lsenta07:05:50

Or learn about the new system that'll deprecate externs soon apparently

lsenta08:05:23

Or.... blaaarg, just keep it simple and do it when you need it 🙂

thheller08:05:08

yeah true ... just don't discard :advanced, always try to keep it working step by step

thheller08:05:07

it is important and should not be left to the last minute

lsenta08:05:17

#clojurescript I have a piece of code that I expect to throw, so I use (is (thrown? js/Error code)). doo, keeps telling me the test fail because that's not the correct error. So I do the following: run the code, catch js/Error and throw it again in a thrown? test.

(deftest test-assoc-throws-invalid-paths
  (go-async-timeoutable S 20000
    (try
      (c/assoc-in! @db [prefix "core" @test-id "throwable/should/fail"] {:k 234})
      (catch js/Error e
        (println "EEE" e)
        (is (thrown? js/Error (throw e)))))))
Here's the output:
Testing konserve-firebase.core-test

EEE #object[Error Error: slashes in token t=throwable/should/fail]

FAIL in (test-assoc-throws-invalid-paths) (:)
Got exception: #error {:message "Error: slashes in token t=throwable/should/fail", :data {}, :cause #object[Error Error: slashes in token t=throwable/should/fail]}
expected: (not (instance? js/Error x))
actual: (not (not true))

lsenta08:05:26

I literally catch the error as a js/Error to have cljs.test tells me it's not a js/Error, how is that possible?

thheller08:05:33

(is (thrown? js/Error e)) not (throw e)

lsenta08:05:12

I throw it again to demonstrate the problem

thheller08:05:27

ah yeah the issue is core.async

thheller08:05:47

it rewrites your code to SSA which looks something like

thheller08:05:48

well no idea really with thrown?

thheller08:05:01

not sure actually. the thrown? macro should still produce a valid try/catch

thheller08:05:04

can't you just do that test sync?

cmal08:05:09

Thank you very much. @lsenta

lsenta08:05:49

Nah, it relies on async feature, this one needs the timeout and all of my other tests will rely on an async lib.

Pablo Fernandez09:05:17

I'm developing three apps at the same item, in ClojureScript, using cljsbuild and I'm putting common parts in a common library. What's the best way to streamline the apps picking up those changes?

Pablo Fernandez09:05:27

I mean, changes to the common library.

lsenta09:05:14

I lein install, which is simple & slow

Pablo Fernandez09:05:17

@lsenta I need to streamline the dev cycle. Slow is not ok. I also found that lein install is not good enough, the other projects wouldn't pick it up. The only thing I found that really works is to deploy the library and let one of the other projects install it, but that's even slower.

lsenta09:05:17

@pupeno what do you mean they don't pick it up?

Pablo Fernandez09:05:48

@lsenta I mean, they don't have the new code. I create a function in the common library, run lein install and the function is not present on the projects that are using the library.

lsenta09:05:04

you do restart the projects after the install?

Pablo Fernandez09:05:39

@lsenta first, that's even slower, but yes, I tried cleaning and recompiling and restarting and nothing.

lsenta09:05:42

- there are a few lib that allows injecting & dealing with jar updates at runtime, not sure what's they're worth - the non-update maybe a cache problem, I know that clojars won't update library without a -SNAPSHOT to avoid issues - have you thought of rsync'ing the files between projects? that's hackish but once you get your cljsbuild auto, rsync will be hard to beat

lsenta09:05:06

you could play with the :source-paths to inject your other libraries

Pablo Fernandez09:05:39

I'd rather not have hackish thing. I would even have to figure out how to get rsync to work on Windows.

Pablo Fernandez09:05:13

I tried playing with :source-paths and I'm not sure if I'm doing something wrong or cljsbuild refuses to read files outside its project.

thheller09:05:38

@pupeno I know nothing about cljsbuild but you could try using lein checkouts and then adding that checkouts/the-lib/src to :source-paths

Pablo Fernandez09:05:19

@thheller lein checkouts require the library to be in the checkouts directory while I need to share the library with three projects.

thheller09:05:51

I thought that was the point yes? I do that frequently

Pablo Fernandez09:05:15

@thheller the point? I'm confused.

thheller09:05:24

~code/the-lib ~/code/project-a ~/code/project-b

thheller09:05:51

project-a/checkouts/the-lib there the-lib is a symlink to ~/code/the-lib

Pablo Fernandez09:05:22

@thheller there are no symlinks on windows.

Pablo Fernandez09:05:29

as far as I know.

thheller09:05:37

hmm there is if you use ubuntu on windows 😉

lsenta09:05:38

I'm pretty sure there are

lsenta09:05:57

not sure if they have the same properties for lein

Pablo Fernandez09:05:31

They are not the horrible .lnk file? interesting.

thheller09:05:42

I have been playing with ubuntu/bash on windows recently and can really recommend it at this point

thheller09:05:50

still a few rough edges but overall very seamless

thheller09:05:17

not sure you'd be interested in that

Pablo Fernandez09:05:48

I'm developing a desktop app for Windows and Mac; I don't want to workaround my target operating system.

thheller09:05:12

yeah I understand, I meant just for the development experience

Pablo Fernandez09:05:26

I'm happy to use Ubuntu for Windows when it's appropriate... although I found it doesn't fix the performance issues I had with Ruby on Windows.

Pablo Fernandez09:05:16

The file system is still an issue. Accessing NTFS from Ubunut is slow and accesing the Ubuntu filesystem from Windows is impossible (unless you consider read-only-write-destroys access an option).

thheller09:05:49

I have a ~/code in ubuntu which is just a symlink to /mnt/c/code

thheller09:05:11

so in windows I work in C:\code and linux ~/code .. works perfectly fine and fast

Pablo Fernandez09:05:16

@thheller that path doesn't sound like Ubuntu for Windows.

Pablo Fernandez09:05:24

Well, Linux for Windows actually.

thheller09:05:43

hmm? which path?

Pablo Fernandez09:05:02

That sounds like an Ubuntu VM.

thheller09:05:11

nope? windows provides that (maybe only in the latest "creator" update?)

Pablo Fernandez09:05:29

I must be misremembering then.

Pablo Fernandez09:05:02

At any rate, if ~/code is a symlink to /mnt/c/code, then your code is on NTFS, which has performance issues (for what I tried to do, not Clojure related).

thheller09:05:36

dunno I can only speak for the latest creator update and everything seemed fast there

thheller09:05:53

building a large CLJS project surely didn't run into any issues

thheller09:05:03

and that writes heavily with caching and all

Pablo Fernandez09:05:36

The issue I had was not read/write but one of the syscalls that asks for metadata about a file.

Pablo Fernandez09:05:11

Rails has auto-code loading so it needs to see which files changed and which ones didn't and one of the syscalls that gets you the last time of change or something like that, on Windows, is horribly slow.

thheller09:05:38

I run my devtools on that which watches all source paths and didn't have any issues

thheller09:05:29

granted the watch has a 2sec built-in delay but that is not window-related but jvm related (also on mac)

Pablo Fernandez09:05:10

I'm sure I posted somewhere my findings but I cannot find it now.

lsenta09:05:30

It's not really that windows is slower but that it doesn't provide hooks to detect file changes. Or maybe it does now.

thheller09:05:31

I'm sure you tested much more extensively than I did ... I have not used ubuntu on windows in anger yet

thheller09:05:21

maybe the windows links work as well

Pablo Fernandez09:05:33

I'm experimenting with pointing source-paths... it seems figwheel sees the files but cljsbuild doesn't (trying to confirm this).

Pablo Fernandez09:05:57

Ha! I think I found the answer.

Pablo Fernandez09:05:26

"../clientcommon/src" doesn't work, but "..\\clientcommont\\src" does.

Pablo Fernandez09:05:12

#notsureifbugorfeature

thheller10:05:34

odd that src/app works but ../clientcommon/src doesn't

tjg14:05:30

Using cljs-http, how do I download a large file? (That is, a POST request where the browser pops up a dialog box & saves the file without reading the whole file into memory.)

tjg14:05:49

(More details: my cljs client calls a Clojure service with a Ring handler whose body is a java.io.File.)

thheller14:05:49

@tjg you can't really do that with ajax requests

thheller14:05:37

you can hack something together using <form ...> tag and submitting that

thheller14:05:11

or POST with cljs-http and have the server give you a url which you can load in the browser

tjg14:05:37

@thheller Thanks! Yeah, that last solution looks good. Will do…

lmergen15:05:07

in clojurescript, using core.async, what would be the best way when you have a sequence of promise chans, to wait until all of them are completed ?

lmergen15:05:30

i’m still trying to wrap my head a bit around the subset of core.async that clojurescript supports 🙂

noisesmith15:05:25

@lmergen wouldn't it suffice to read from them all in a row?

noisesmith15:05:52

since each one would park until realized, and then return immediately if realized

lmergen15:05:35

hmmm i need to combine all the results, like a zip operation… perhaps a merge and a reduce ?

noisesmith15:05:45

that would work, so would a normal reduce, where the last arg is the list of promise-chans, and the reducing function reads from them and uses the result

lmergen15:05:39

reading can only be done inside a go block, right… which returns a channel itself ?

noisesmith15:05:05

you would reduce inside a go block - anything reading from channels will be in a go block

lmergen15:05:50

ok, i’m going to try this approach, thanks!

noisesmith15:05:28

oh wait - I think you need merge and async/reduce

noisesmith15:05:36

I forgot that the function inside the reduce loses the go context

lmergen15:05:03

i kind of ran into that

noisesmith16:05:14

merge fails because promises don't close

lmergen16:05:54

yeah i kind of worked around that

noisesmith16:05:16

+user=> (let [chans (repeatedly 10 >/promise-chan)] (doseq [c chans] (>/>!! c 42)) (>/<!! (>/go (>/<! (>/reduce + 0 (>/merge (map #(>/take 1 %) chans)))))))
420

noisesmith16:05:31

where clojure.core.async :as >

noisesmith16:05:10

much cleaner with multiple lines and a ->> macro

lmergen16:05:45

but that uses <!! right, which doesn’t exist in clojurescript

noisesmith16:05:13

only for repl friendliness

noisesmith16:05:20

its on the outer edge of the code

noisesmith16:05:38

(doesn't look like "outer" because it's a one liner, but it is)

noisesmith16:05:54

fairly straightforward translation to ->>

(let [chans (repeatedly 10 >/promise-chan)]                                     
  (doseq [c chans]                                                              
    (>/>!! c 42))                                                               
  (>/<!!                                                                        
   (>/go                                                                        
    (->> chans                                                                  
         (map #(>/take 1 %))                                                    
         (>/merge)                                                              
         (>/reduce + 0)                                                         
         (>/<!)))))  

noisesmith16:05:11

this one works but looks crazy

(let [chans (repeatedly 10 >/promise-chan)]                                     
  (doseq [c chans]                                                              
    (>/>!! c 42))                                                               
  (->> chans                                                                    
       (map #(>/take 1 %))                                                      
       (>/merge)                                                                
       (>/reduce + 0)                                                           
       (>/<!)                                                                   
       (>/go)                                                                   
       (>/<!!)))

noisesmith16:05:35

I think there's a way to do this with pipeline and the take transducer

noisesmith16:05:49

it's an interesting problem - there feels like there should be a higher level form that abstracts this pattern of collection / channel usage, but I don't see one that works

thheller16:05:27

alts! works well for such things

noisesmith16:05:37

@thheller but all the promise chans all have data available once realized

noisesmith16:05:41

so it doesn't fit

noisesmith16:05:03

you need to do the map of take 1 somewhere, and to me that's the messy part

lmergen16:05:21

hmmm, what’s the reason a promise chan doesn’t close once it’s fulfilled ?

noisesmith16:05:47

because it's meant to infinitely return the same result every time it is read

noisesmith16:05:54

maybe promise chan isn't the thing you want?

thheller16:05:06

aren't you supposed to close! it? (never used promise-chan)

thheller16:05:02

but alts! still works just don't pass in the chan once you read something from it

noisesmith16:05:48

no - it keeps returning the same result even after close

Clojure 1.9.0-alpha15
+user=> (require '[clojure.core.async :as >])
nil
+user=> (def c (>/promise-chan))
#'user/c
+user=> (>/put! c 42)
true
+user=> (>/<!! c)
42
+user=> (>/<!! c)
42
+user=> (>/close! c)
nil
+user=> (>/<!! c)
42

noisesmith16:05:23

chans keep returning data if buffered even after closed, promise-chans continue to have data buffered forever

noisesmith16:05:03

oh, looping with alts and removing the read channel would work, but imho it would be even messier than what we had already

noisesmith16:05:27

(explicit looping logic and all)

lmergen16:05:46

so the promise chan is actually making all this more difficult, otherwise it was just a matter of merge and reduce

lmergen16:05:54

i can work around that

thheller16:05:51

(defn wait-for-all [chans]
  (go (loop [chans-to-read
             (set/map-invert chans)
             result
             {}]

        (if (empty? chans-to-read)
          result
          (let [[v ch] (alts! (keys chans-to-read))]
            (recur
              (dissoc chans-to-read ch)
              (assoc result (get chans-to-read ch) v))
            )))))

(let [foo (async/promise-chan)
      bar (async/promise-chan)]

  (>!! foo :foo)
  (>!! bar :bar)

  (prn (<!! (wait-for-all
              {:foo foo
               :bar bar}))))

lmergen16:05:00

man, clojurescript sure makes flaws in your async logic more obvious

thheller16:05:14

could use another collection type, doesn't have to be a map

lmergen16:05:14

you can’t get away with using <!! 🙂

thheller16:05:39

@lmergen it doesn't need <!! just easier to test in the REPL 😉

lmergen16:05:52

no i was talking about clojurescript 🙂

noisesmith17:05:07

@thheller yeah, that's about what I expected it to look like - it is a lot more code, and much more imperative, than the map / merge / reduce

thheller17:05:30

@noisesmith you put the wait-for-all function somewhere and never look at it again

noisesmith17:05:42

but wait-for-all is the part I don't like

lmergen17:05:58

wait-for-all looks a lot like a barrier…

lmergen17:05:38

seems like that is the missing construct 🙂

noisesmith17:05:57

@thheller my criterion here is the points in tim baldridge's talk at the last clojure west, removing state and io from core.async code by using higher order functions

thheller17:05:19

@lmergen that is only a basic example. actual code wants to handle timeouts and such

thheller17:05:25

there is no easy general solution for that

thheller17:05:37

also what do you do if one chan fails

thheller17:05:52

as soon as you want to account for failures a general wait-for-all doesn't work

lilactown20:05:35

i'm trying to figure out a good way to strip HTML received via AJAX. is there a lib out there that i can use?