Fork me on GitHub
#hoplon
<
2015-11-07
>
thedavidmeister05:11:57

does anyone else get errors like

thedavidmeister05:11:00

WARNING: Required namespace not provided for cljs.estimate-work.vectors null

thedavidmeister05:11:07

that kind of thing?

thedavidmeister05:11:23

I’m trying to google it but not getting much relevant info

micha05:11:15

that looks like what happens when you (:import [foo.bar Baz]) maybe

thedavidmeister05:11:39

would :require have a similar problem?

micha05:11:00

i've seen something similar i think when using js things via foreign libs

micha05:11:17

where are you getting the error?

thedavidmeister05:11:25

i get it in different places actually

micha05:11:33

in the compiler or in the browser?

thedavidmeister05:11:46

when i run (cljs :optimizations :advanced)

thedavidmeister05:11:05

i pretty much always get it for files that have (page foo) instead of (ns foo)

thedavidmeister05:11:30

sometimes, i’m not sure when because it’s not 100% of the time

micha05:11:37

estimate-work

thedavidmeister05:11:41

i get it for things that i’m requiring inside the page

micha05:11:43

is this a namespace you made?

thedavidmeister05:11:48

name of the project

micha05:11:52

that could be the problem

micha05:11:15

you want to make your namespaces so they don't conflict with other people's namespaces

micha05:11:35

the is a bad idea

thedavidmeister05:11:38

what am i conflicting with?

micha05:11:46

cljs is owned by clojurescript

thedavidmeister05:11:51

well right now i’m trying to get tests running

thedavidmeister05:11:58

so this is the third place i’m seeing it

micha05:11:06

yeah i'd change that

thedavidmeister05:11:21

i couldn’t find good examples of how to structure my folders/files/namespaces for testing

micha05:11:23

the namespace system is devised to help you prevent collisions

micha05:11:28

like the domain name system

thedavidmeister05:11:37

everything i saw just said “oh, assuming you’re using a folder called test add this config…"

thedavidmeister05:11:48

yeah but, how do namespaces work for testing?

micha05:11:19

like suppose i have a namespace adzerk.foop for instance

thedavidmeister05:11:19

well, when I use phpunit with PSR I just mirror the namespace of the classes i want to test

thedavidmeister05:11:31

but with tests in the namespace or something

micha05:11:37

i make a second namespace for tests related to that namespace

micha05:11:50

i will usually name it adzerk.foop-tests

micha05:11:53

but it doesn't really matter

thedavidmeister05:11:07

but don’t namespaces have to match folder structure to get included properly?

micha05:11:20

yes they do

thedavidmeister05:11:45

so if i have a directory in root called src/cljs with .cljs files inside it

thedavidmeister05:11:03

src/cljs/estimate_work/vectors.cljs

thedavidmeister05:11:14

has (ns estimate-work.vectors) in it

micha05:11:33

so the first thing you want to do is not clobber the cljs namespace

thedavidmeister05:11:36

then how do i match up a testing directory/file structure to test thigns in that

micha05:11:41

you can just change the first segment to some other name

micha05:11:53

then what i usually do is this

thedavidmeister05:11:03

that came from the hoplon castra template though

thedavidmeister05:11:09

i don’t think i set that up

micha05:11:28

one second

micha05:11:00

which template?

micha05:11:21

well i can tell you how i do it

thedavidmeister05:11:27

i looked for a template specifically to avoid this kind of noob mistakes 😛

micha05:11:47

like suppose you want to organize your source code in such a way that tests are separate from the working code

micha05:11:01

and maybe you have different folders for different types of source code

micha05:11:12

like scss in one directory and cljs in another or something

micha05:11:26

so i would make a directory "src"

micha05:11:46

and in there "cljs", "test", and "scss"

thedavidmeister05:11:00

so, test inside src

thedavidmeister05:11:02

that’s one thing i did wrong

micha05:11:06

i mean you don't have to

micha05:11:13

sometimes i put it at the same level as "src"

micha05:11:18

that would work also

micha05:11:24

the key is this though

micha05:11:45

(set-env! :source-paths #{"src/cljs", "test", "src/scss"})

thedavidmeister05:11:46

my gf actually put sass in the top level to make it easier for her to compile into assets

thedavidmeister05:11:50

but i don’t think that matters here

thedavidmeister05:11:10

:resource-paths #{"assets" "src/clj"}
  :source-paths   #{"src/cljs" "src/hl" "test”}

micha05:11:14

the idea here is that when you configure the :source-paths and :resource-paths

micha05:11:19

it will overlay all those directories

micha05:11:47

and the files in them will be resolved relative to the overlayed directories

micha05:11:01

so like supose i have my adzerk.foop namespace

thedavidmeister05:11:17

ok, i just checked

thedavidmeister05:11:19

lein new hoplon-castra foo

thedavidmeister05:11:26

you’ll see what i ended up with

thedavidmeister05:11:51

you get src/clj src/cljs and src/hl

micha05:11:18

.
├── sass
│   └── _application.scss
├── src
│   └── cljs
│       └── adzerk
│           └── foop.cljs
└── test
    └── adzerk
        └── foop-test.cljs

micha05:11:40

this is what i'd make

micha05:11:39

i just installed the template

thedavidmeister05:11:55

so i didn’t mean to clobber cljs

thedavidmeister05:11:59

if that’s what i did...

micha05:11:02

if you pull a fresh copy of the template

micha05:11:09

then do boot show -f in there

micha05:11:18

that will show you the overlayed files

thedavidmeister05:11:30

just x-referencing your nice diagram against my current setup

thedavidmeister05:11:35

shuffling a few things around

micha05:11:45

the diagram was just one possible way

micha05:11:00

the important thing is to understand this overlaying business

micha05:11:12

boot show -f is your friend

thedavidmeister05:11:20

but i wanted to start with one working example

thedavidmeister05:11:26

then experiment from there

micha05:11:38

it will show you how boot is assembling the various directories

thedavidmeister05:11:52

oooh my tests ran without errors!

thedavidmeister05:11:55

the tests failed

micha05:11:10

then when you move things around you can use boot show -f to see how your changes affected the way the tasks see the files

micha05:11:58

~/zzz/foop
barp $ tree .
.
├── assets
│   └── app.css
├── boot.properties
├── build.boot
├── README.md
└── src
    ├── clj
    │   └── foop
    │       ├── api.clj
    │       ├── core.clj
    │       └── handler.clj
    ├── cljs
    │   └── foop
    │       └── rpc.cljs
    └── hl
        └── foop
            └── index.cljs.hl
8 directories, 9 files
~/zzz/foop
barp $ boot show -f
app.css
foop
├── api.clj
├── core.clj
├── handler.clj
├── index.cljs.hl
└── rpc.cljs

thedavidmeister05:11:32

tdm:estimate-work thedavidmeister$ boot show -f
app.css
estimate_work
├── api.clj
├── core.clj
├── elements
│   ├── bucket.cljs.hl
│   ├── estimate
│   │   └── fixed.cljs.hl
│   ├── form
│   │   └── atom
│   │       └── textfield.cljs.hl
│   ├── nav
│   │   └── support.cljs.hl
│   └── outer-dom.cljs.hl
├── handler.clj
├── index.cljs.hl
├── rpc.cljs
├── style-guide.cljs.hl
├── vectors.cljs
└── vectors_test.cljs
ie.css
print.css
screen.css

thedavidmeister05:11:52

here i thought you drew me that nice diagram by hand

thedavidmeister05:11:56

but it was all the machine

micha05:11:57

notice how boot is collecting src/clj/foop/, src/cljs/foop/, and src/hl/foop/*

micha05:11:19

in show -f it appears to be just one directory

micha05:11:06

if i wanted to make some tests i can put them anywhere, as long as i add the correct entry to :source-paths or :resource-paths

thedavidmeister05:11:40

yeah sure, so comparing tree against show -f seems like something important to debugging from time to time

micha05:11:18

yes exactly

thedavidmeister05:11:20

so how do i know what goes in source paths and what goes in resource paths?

micha05:11:41

things in resource paths will end up in the "final artifacts"

micha05:11:49

i.e. in a jar file or in the target directory

thedavidmeister05:11:53

because it was recommended to me to move src/clj from source to resource as part of deploying a war to heroku

thedavidmeister05:11:05

but i didn’t really get why at the time

micha05:11:20

well it has to do with the jvm really

micha05:11:32

boot is a program that's running in the jvm

micha05:11:41

building a program that will run in a different jvm

micha05:11:05

actually let me start over

micha05:11:21

there are files that are source files

micha05:11:30

that you will be compiling and doing things to

micha05:11:50

some of those you want to propagate to whatever the final artifact is that you're building

micha05:11:03

i.e. the war file, or a zip file, or files in the target directory

micha05:11:27

but some of the source files that will be part of the build process are not supposed to end up on the webserver

micha05:11:54

like maybe your uncompiled cljs namespace files are not intended to be served from the web server

micha05:11:01

those 50 lines explain the whole thing

thedavidmeister06:11:15

so there’s a bit of assumed knowledge about the JVM in there

thedavidmeister06:11:30

what does the AOT do? for example

thedavidmeister06:11:59

other than stopping a file from being a resource

micha06:11:23

aot means "ahead of time"

micha06:11:45

referring to the practice of compiling clojure namespaces into byte code

micha06:11:21

i mean "into bytecode"

micha06:11:42

clojure is a dynamic language

micha06:11:48

it contains its own compiler

micha06:11:13

and most of the time you run clojure in such a way that it compiles your program as it runs

micha06:11:37

but sometimes you need to precompile some things into bytecode before the program can run

micha06:11:58

like if you're compiling things that need to expose Java classes

thedavidmeister06:11:10

this doesn’t apply to clojurescript though does it?

thedavidmeister06:11:16

you wouldn’t need to aot JS?

micha06:11:30

clojurescript doesn't have a self hosting compiler really

micha06:11:36

well it has a very new thing

micha06:11:51

but historically it hasn't been able to compile itself

micha06:11:05

so you have no choice but to compile your clojurescript program ahead of time

micha06:11:23

you can't send cljs source files to the browser and compile them there

thedavidmeister06:11:34

yeah i think that’s what i meant

micha06:11:34

clojure on the other hand can and does do this all the time

thedavidmeister06:11:38

you don’t really have a choice

micha06:11:46

most clojure jar files just contain .clj files

micha06:11:49

the source

thedavidmeister06:11:49

because it always has to be JS before it’s useful

thedavidmeister06:11:27

right, so why would i not want to aot before deploying something?

thedavidmeister06:11:43

wouldn’t it be a bit faster to get the byte code as part of making my war?

thedavidmeister06:11:57

faster to run the program i mean

micha06:11:01

well you can't compile dynamic programs ahead of time

micha06:11:18

there are many things you can't do in cljs because this

micha06:11:25

things that you can do in clojure

micha06:11:30

cljs has no eval

micha06:11:37

which is an important part of lisp

micha06:11:48

so in general you don't want to AOT

thedavidmeister06:11:44

so aot = no eval

thedavidmeister06:11:56

is that macros as well?

thedavidmeister06:11:58

so when you say exposing java classes, you don’t mean just calling methods on the classes do you?

thedavidmeister06:11:09

because that would come up a bunch i assume

micha06:11:17

i mean like if i make a clojure library that i want to expose to java developers to use in their java programs

micha06:11:34

and expose it to them in such a way that they don't even know it's not java

thedavidmeister06:11:45

for going back the other way

micha06:11:51

i'd have to create concrete java classes then, and interfaces, etc

thedavidmeister06:11:31

yeah, like, if it wasn’t so crazy i could tell someone i wrote the JS myself that cljs made

thedavidmeister06:11:43

and they’d have no way to prove i didn't

thedavidmeister06:11:10

does the aot for java actually make code that someone would find legible?

thedavidmeister06:11:13

or is that not the point?

micha06:11:23

no, it doesn't generate java source code

micha06:11:29

it generates bytecode

micha06:11:46

it generates the stuff you'd get if you ran javac on a .java file

micha06:11:57

you get a .class file, which contains binary bytecode

thedavidmeister06:11:25

mmmk, i really am not planning to learn much java if possible

thedavidmeister06:11:30

so maybe i’ll leave that for now

micha06:11:44

haha i don't blame you

thedavidmeister06:11:12

i don’t love needing to learn about the JVM to get clojure stuff working

micha06:11:45

it's unavoidable

micha06:11:58

at least the basics

thedavidmeister06:11:06

this document helped a bunch

thedavidmeister06:11:15

i didn’t really know about the classpath or the different types of files

thedavidmeister06:11:52

any idea why assets ends up listed as a resource-path instead of asset-path?

thedavidmeister06:11:12

also, am i right in saying, because the .cljs files get compiled to JS before being dumped in target/ that is what makes them source-path and not resource-path?

thedavidmeister06:11:03

and because the src/clj ends up powering the server backend for castra, and we don’t want to AOT it, to keep all the lispy goodness working, we put it in resource-path?

thedavidmeister06:11:30

and because we don’t actually ever want our tests to hit production, because that’s pointless, we leave them in source-path?

micha06:11:59

yes that's corect

thedavidmeister06:11:04

@meeli: micha gave me some really good info about the file structure ^^

thedavidmeister06:11:09

you should read over it when you get a chance

micha06:11:55

when tasks create files they generally create them as resource files

thedavidmeister06:11:05

@micha: is there any benefit to using asset-path over resource-path? or is it just easier to use resource-path everywhere you might want something to end up as an artifact

micha06:11:18

because it's assumed that you're creating them to output them somewhere

thedavidmeister06:11:21

it looks like assets are just less flexible versions of resources

micha06:11:30

asset-path only really exists for completeness

micha06:11:45

since there are 4 permutations i figured we should give them all names

micha06:11:59

but i've never found a good use for asset-paths myself

thedavidmeister06:11:30

well if you were using something other than boot to generate the assets?

thedavidmeister06:11:37

like rake or something?

micha06:11:19

i don't think i have anything like that myself

micha06:11:34

we do run sassc for example to compile scss

thedavidmeister06:11:36

pretty sure we’re still using ruby to do the sass compiling

micha06:11:41

but we do it from a boot task

thedavidmeister06:11:44

haven’t looked at other ways to get sass compiling yet

thedavidmeister06:11:51

but i did see there was a lein sass

thedavidmeister06:11:54

just haven’t migrated to it yet

thedavidmeister06:11:01

one thing at a time

thedavidmeister06:11:21

but you know, wouldn’t image files be assets?

thedavidmeister06:11:37

boot wouldn’t really be doing anything with the images other than moving them into place

micha06:11:56

yeah that could be

thedavidmeister06:11:04

but at the same time

thedavidmeister06:11:13

you just limit yourself by making it an asset not a resource

micha06:11:33

yeah that's what i was thinking too

thedavidmeister06:11:41

is it somehow more efficient under the hood?

thedavidmeister06:11:48

or if not now, could it be in the future?

thedavidmeister06:11:16

if boot knew it would never need to do certain things with your asset files, maybe it wouldn’t need to load as much into memory, or something...

micha06:11:25

yeah that's true

micha06:11:49

it could save some cycles if you have a ton of files

thedavidmeister06:11:07

well it would just really need to keep a reference to where the file lives on the file system

thedavidmeister06:11:21

then right at the end, just copy and paste from the reference to the target

thedavidmeister06:11:05

i wouldn’t bother implementing that unless you’d profiled some real issues on a real project >.<

micha06:11:41

i've been working on profiling some fileset related things the past few days

micha06:11:03

java IO is really pretty fast

micha06:11:14

they did a good job with it

thedavidmeister06:11:24

i did blow up the 4GB memory limit while trying to build pretty much the base template on circle ci

thedavidmeister06:11:30

dunno if that’s related to what you’re doing

micha06:11:05

that's odd

thedavidmeister06:11:18

not sure if boot even finished pulling down the dependencies

thedavidmeister06:11:29

it freaked out and i couldn’t even download the artifacts containing the memory info

micha06:11:33

there were some clojars issues

micha06:11:40

this week

thedavidmeister06:11:50

oh, maybe that impacted me

thedavidmeister06:11:59

i got my project deploying to heroku

thedavidmeister06:11:07

but it’s still manual, wanted merges to master to auto deploy

thedavidmeister06:11:17

had to shelve it for now until i learn more about what’s going on

thedavidmeister06:11:26

was getting errors about clojars not being git repos or something...

thedavidmeister06:11:45

and circle ci maxed out memory before i got very far through seeing if they could help

micha06:11:46

is circle ci similar to travis?

thedavidmeister06:11:56

actually i’ve used travis more

thedavidmeister06:11:01

from what i understand

micha06:11:06

we do stuff on travis with boot all the time

thedavidmeister06:11:09

ci runs your tests, then optionally does a deploy task as well

thedavidmeister06:11:16

yeah travis worked for me so far

thedavidmeister06:11:34

in theory circle ci is travis + deploy

thedavidmeister06:11:48

in practice, it looks like they have pretty different setups/limits on the hardware/environments

thedavidmeister06:11:16

so i’m sure for lots of people it’s awesome, didn’t work out for me this time though...

micha06:11:35

i would hope that boot would use less memory than say leiningen

micha06:11:46

because it's running in a single JVM

thedavidmeister06:11:20

i really don’t know

thedavidmeister06:11:28

i’m just throwing shit at the wall to see what sticks

thedavidmeister06:11:42

too noob to have an informed opinion about what really should or shouldn’t work

micha06:11:02

well have a good rest of the night simple_smile

micha06:11:13

i'm off to sleep

thedavidmeister06:11:28

if you’re curious and profiling things anyway

thedavidmeister06:11:32

+dependencies:
+  post:
+    - wget -O boot 
+    - chmod 755 boot
+    - ./boot -V
+    - ./boot checkout
+test:
+  override:
+    - ./boot test

thedavidmeister06:11:37

i stuck that in my circle.yml

thedavidmeister06:11:06

@micha: see ya, thanks again for helping out!

alandipert20:11:51

@micha can you describe how .inc.js used to work in hoplon5? i wonder if a task to do that on h6 would be cool

micha20:11:41

.inc.js files were added to the top of the compiled js file, in dependency order if they came from dep jars, lexical order for .inc.js files in the fileset

micha20:11:56

but it required a scan of the jars which can get really slow

micha20:11:09

i think the problem is now completely solved with cljsjs

alandipert20:11:10

i think it's handier for 1-off jq plugins from an vendor/ dir

alandipert20:11:21

then if you like them you may cljsjs them

micha20:11:29

we should make a task that cljsjses them

micha20:11:42

like generates the deps.cljs file

alandipert20:11:48

i was looking into that, can thye be their own externs?

micha20:11:56

you don't need externs

micha20:11:02

or yes, they can be their own externs

micha20:11:10

but that's like only partially successful

micha20:11:26

just don't do :advanced i would say

micha20:11:36

unless you are ready to invest your time in a real externs file

micha20:11:54

or use a macro lib that gets around the mangling

micha20:11:16

like transforms interop to the aget/`aset` forms

alandipert20:11:01

it's nice to have your cljs :advance compiled but then load up the jq libs dynamically

micha20:11:58

it's not the jq that gets munged of course

micha20:11:06

it's the references in cljs land to them