Fork me on GitHub
#clojure
<
2016-01-13
>
arohner00:01:16

I have some kind of unicode problem: some other system has files that look like they were stored improperly w.r.t to unicode: I have a file that contains "If You Don\342\200\231t Know”. Is there some string encoding that will parse that correctly? All variants of UTF-8, ASCII and ISO-85whatever haven’t worked so far

aaelony01:01:19

I've had that problem in the past and the only way around it that worked for me was finding the offending characters and replacing them with safe placeholders that indicate what was there...

meow01:01:34

@aaelony: that sounds like dialog from a cop show on tv

aaelony01:01:03

I think if you have a data source with a lot of raw chat text, crazy stuff shows up that causes reasonable tools to barf or complain

meow01:01:48

you should see the crazy sh*t I'm posting in #C0J20813K

meow01:01:40

well, in the developer instance of it, not the slack channel about it

meow01:01:49

off-topic, my bad

arohner01:01:08

@aaelony: I don’t think it’s a raw chat problem. it’s a user posts valid unicode, some other system handles unicode improperly, and writes it back out as ASCII or something, and now I want to re-parse it as the original unicode

arohner01:01:24

seems like a common enough problem that there should be a library, if I knew what to google for

meow01:01:46

not sure if either of those will help but they are useful

domkm02:01:24

Is there a way to test if an object can be printed in Clojure and read in ClojureScript? For example, #inst "2016" can be read in ClojureScript but #error {...} cannot.

kenrestivo02:01:01

[6[6[6[6[6[6/quit

jonahbenton04:01:17

@arohner: hey allen, looks like that's the octal escape sequence for http://graphemica.com/%E2%80%99.

jonahbenton04:01:18

so replace \ with 0, and get (Integer/parseInt "0342" 😎 -> 226

jonahbenton05:01:55

bytes 226 128 153, in a byte array, passed to (String. "UTF-8") yields the quote

jonahbenton05:01:22

would be a good library

madstap05:01:27

One of the arities of if-let is [bindings then else & oldform]. What is the point of oldform?

madstap05:01:54

It just seems to throw an exception

hans05:01:08

it is there for error reporting purposes

hans05:01:18

so that the error message is clear about what you've done wrong.

hans05:01:55

i'd argue that "oldform" is not a good argument name, though.

madstap05:01:03

it's nice that they thought about error messages at least in this case, but that is one confusing signature

madstap05:01:58

thanks for clearing that up, I've pondered that a few times and couldn't really find anything with basic googling

jethroksy08:01:54

I have a scenario here that I need help with

jethroksy08:01:13

I'm using boot, yesql and environ, although I suspect boot or lein the issue is the same

jethroksy08:01:56

So I would like to pull the correct subname or dburl from environ based on the environment

jethroksy08:01:04

So I have in foo.db,

(def conn {... (env :dbsubname)})
(defqueries "queries.sql" { :connection conn})

jethroksy08:01:26

The problem is before environ serves up things, my program evaluates (env :dbsubname) to be nil

jethroksy08:01:39

My queries will then be invalid

jethroksy08:01:36

So what's the best way to trigger the re-evaluation of conn and the defqueries macro?

martinklepsch08:01:29

@jethroksy: I recommend reading the environ code to see what it's doing. It's very little code iirc

jethroksy08:01:48

Thanks, I've taken a look, but that didn't help

jethroksy09:01:31

It turns out to be a compile vs runtime issue, and there's no easy way to get environ and yesql to work together

apviitanen09:01:01

@jethroksy: You can pass the connection to yesql when running the query (since yesql 0.5 I think). This might not be documented well though..

jonahbenton12:01:40

@jethroksy: you don't have to pass the connection spec at query definition time.

jonahbenton12:01:50

just pass the spec as the first parameter to the query functions themselves

jonahbenton12:01:30

also, may want to take a look at Component https://github.com/stuartsierra/component or Mount https://github.com/tolitius/mount to help manage runtime lifecycle of resources like database connections

jethroksy12:01:46

@apviitanen: @jonahbenton just looked at the source and it seems like its possible. thanks!

joelkuiper12:01:46

is there a take-while that takes until and including pred becomes false?

joelkuiper12:01:35

e.g. (take-while neg? [-2 -1 0 1 2 3]) => [-2 -1 0] instead of [-2 -1] ?

joelkuiper12:01:31

I guess I can use split-with

joelkuiper12:01:03

hmm maybe not

joelkuiper12:01:15

I want to keep the laziness of not having to traverse the entire sequence

joelkuiper12:01:28

this’ll do

(defn take-while-and-n-more
  [pred n coll]
  (lazy-seq
   (when-let [s (seq coll)]
     (if (pred (first s))
       (cons (first s) (take-while-and-n-more pred n (rest s)))
       (take n s)))))

meow13:01:37

intersting

meow13:01:50

oh, shit, is this a non-smoking channel - my bad

meow13:01:24

it was just weed

meow13:01:51

oops, that was for #C0FQERS0H

meow13:01:01

wrong channel

hans13:01:52

The :mirrors configuration in my :user profile is not picked up when I use lein war, can anyone enlighten me why?

hans13:01:57

I guess I'll need to arrange for clojars to be down and hold that situation hostage to get the solution 😕

hans13:01:09

i. was. only. joking.

hans14:01:53

@tcrawley: yes, it seems like that. unfortunately, as there does not seem to be a solution.

tcrawley14:01:01

you may be able to replace the repos entirely: https://github.com/clojars/clojars-web/wiki/Mirrors

hans14:01:57

Oh, I'll try that, thanks!

hans14:01:13

It does not help. Depressing.

tcrawley14:01:28

@hans: why are you needing to hit a mirror?

hans14:01:18

@tcrawley: We have proprietary software in our repository and by the way of setting a mirror, I'm making that software available to projects that need it.

hans14:01:38

@tcrawley: E.g. the datomic-pro jars and stuff we've written internally.

tcrawley14:01:39

seems like you would want that to be another repository, not a mirror. unless it is also mirroring clojars for you

hans14:01:53

It mirrors clojars as well.

hans14:01:21

But if a separate repository would work better for the proprietary stuff, we can move to that.

tcrawley14:01:51

is this the lein war task, or lein immutant war? I know the latter doesn't respect mirrors

hans14:01:45

It is lein ring war.

hans14:01:20

I would fix lein-ring if I understood enough of it, but I'm rather overwhelmed.

tcrawley14:01:57

it seems odd that replacing the repos fully doesn't work either - what error do you see? just a failure to resolve the jars?

hans14:01:20

Right, and the warning:

(:repositories detected in user-level profiles! [:user]
See 

tcrawley14:01:44

I'd think that the only way fully replacing the repos wouldn't work is if the war task hardcoded a repo

tcrawley14:01:03

right, you'll get that warning, but can ignore it (at least for purposes of debugging here)

tcrawley14:01:32

does it fail to find your proprietary jars, or all jars?

hans14:01:29

Just the proprietary ones. I've put this in my profiles.clj:

:repositories ^:replace [["central" {:url ""
                                             :repo-manager true}]
                                 ["clojars" {:url ""
                                             :repo-manager true}]]
        :plugin-repositories ^:replace [["central" {:url ""
                                                    :repo-manager true}]
                                        ["clojars" {:url ""
                                                    :repo-manager true}]]
and also made the same entry for :plugin-repositories in the project.clj file in question.

tcrawley14:01:04

what is the dependency spec in project.clj for one of the failing jars?

hans14:01:14

@tcrawley: [lambdawerk/fixtures "0.10.0"]

hans14:01:35

indeed it does, and it is downloaded when i use lein deps

hans14:01:11

it is an internal system, which is why the dns name does not exist for you

tcrawley14:01:14

what is the error message from lein when it's not found?

hans14:01:32

Could not find artifact lambdawerk:fixtures:jar:0.10.0 in central ()
Could not find artifact lambdawerk:fixtures:jar:0.10.0 in clojars ()
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
Compilation failed: Could not resolve dependencies

tcrawley14:01:34

right, I figured it was internal, just making sure things are in the correct place

tcrawley14:01:53

@hans: I don't know where it's picking up the primary urls then. the war task doesn't do any dep resolution on its own

joelkuiper14:01:54

@alexmiller: thanks! Just what I was looking for simple_smile

hans14:01:23

@tcrawley: thanks for caring. i'm hitting a brick wall with this and will probably have to dig into leiningen and/or lein-ring to fix it 😞

tcrawley14:01:43

sorry I don't have a solution for you :(

martinklepsch15:01:09

is there a way to check/validate data against a prismatic schema definition that does not throw an exception?

jonahbenton15:01:58

@hans curious- why mirror instead of just having nexus proxy clojars/maven central for you?

martinklepsch15:01:46

@hans: Thanks! should look more closely simple_smile

hans15:01:34

@jonahbenton: that is what nexus does for us, and i think that :mirrors is the way to tell leiningen what the proxy would be.

danielgrosse15:01:11

I have an XML Structure, which has nested childrens. I want to differ between childs, which have a children and those who haven't. How could I achive this?

danielgrosse15:01:01

Yes. I worked through it, but didn't get it. My problem is the depth of the structure.

danielgrosse15:01:10

<data title="Title">
    <children hidden="false">
        <title><![CDATA[Innenbereiche]]></title>
        <file>tui_5852</file>
        <children hidden="false">
            <title><![CDATA[Service]]></title>
            <file>tui_5852</file>
            <children hidden="false">
                <title><![CDATA[Rezeption / Atrium]]></title>
                <type>360</type>
                <sound>background.mp3</sound>
                <file>tui_5852</file>
                <map>map_deck_05_pier</map>
            </children>
        </children>
    </children>
</data>

danielgrosse15:01:48

There could be childrens at any level,

danielgrosse15:01:17

I want to build a edn structure out of it.

hans15:01:47

what do you have so far?

jonahbenton15:01:00

@hans did you try :omit-default-repositories?

hans15:01:31

@jonahbenton: no - how would that help? my workaround right now is that the dependency that lein ring war cannot load is actually not needed during war construction, so i moved it into the appropriate profile. but the problem will resurface sooner or later, so i'm interested in fixing it.

hans16:01:18

from my cursory research, :omit-default-repositories is from before ^:replace existed and does the same thing (i.e. does not fix the problem)

jjttjj16:01:24

if i'm dealing with core.async mults, is it more idiomatic to mass around a mult to various functions that need it and tap it within the function, or should i just tap a new chan at a higher level and only deal with actual chans in the lower level functions, or am i just worrying too much about this

danielgrosse16:01:34

I also tried the way of korny, with

(->> reader
          parse
          :content
          (filter...))
But didn't found a way to find the correct children in depth

jonahbenton16:01:17

@hans apologies, i was looking at an old project. your problem is that at war packaging time that proprietary dependency can't be found?

hans16:01:28

@danielgrosse: so you want to create a hierarchy from an arbitrarily nested xml hierarchy? it sounds as if you'll need to walk the structure recursively, constructing the new structure during the traversal. i often use clojure.walk for such tasks, but there are other ways i guess.

hans16:01:05

@jonahbenton: correct - for some reason, the :mirrors declaration in my :user profile is not picked up during lein ring war

jonahbenton16:01:37

hmm, do you need to use lein ring uberwar?

hans16:01:51

no, i really don't want to pull in all dependencies.

hans16:01:29

@jonahbenton: i've seen that place in the code, but i could not devise the command line that you gave from it. let me try.

hans16:01:02

@jonahbenton: no dice, but thanks

jonahbenton16:01:23

any clues from increased verbosity: DEBUG=true lein with-profile +user ring war

hans17:01:03

nope, nothing 😞

jonahbenton17:01:13

one other shot in the dark- does the dep exist in your local fs repo, under .m2?

rfhayashi18:01:32

hi, when using clojure.java.shell/sh, if i read the doc correctly, you can only get the whole output after the command finishes, i'd like to consume the output as the command runs, i found this library: https://github.com/Raynes/conch, it seems to support that, any other suggestions?

bitsynthesis19:01:57

hey folks. if i'm defining a new class in a clojure namespace with gen-class, and i want to utilize that class in another namespace... is there a better way than both requiring the clojure namespace AND importing the generated class?

bitsynthesis19:01:52

got answer from IRC: only require is strictly necessary, import just allows you to refer to the class unqualified

bitsynthesis19:01:05

so if referring to the class fully qualified is acceptable, import is not needed

derwolfe20:01:59

ztellman: I’m feeling a bit dumb; I’m using manifold streams and deferreds and having trouble getting a thing that uses connect-via to work. The question I have is: does it make any sense for a function that will only be accessed using connect-via to return anything? I have a function that does some asynchronous http calls; inside of an manifold.deferred/chain call. I am noticing that unless I deref the result of the http call (in effect making it block), the http call never returns its body. I’m working on a small reproducible test now to help explain the issue.

jtackett21:01:41

What is the best way to set a static IP address for a clojure app when using heroku? I need to whitelist my IP address for queries to a database

jtackett21:01:02

is there a good step by step guide for that?

jtackett21:01:21

looks like proximo doesn’t work with JVM languages

codefinger21:01:00

it does, but differently. promxio is kind... not as good as the others

codefinger21:01:17

i don't think i have any in clojure.

codefinger21:01:29

the docs for each add-on might have some

jtackett21:01:39

they don’t have anything clojure specific

jtackett21:01:37

@codefinger: tough to translate OO to clojure

jtackett21:01:53

do you have to change code at each request?

jtackett21:01:08

or is there some kind of middleware or wrapper so that it will handle the routing to proxy for you

codefinger21:01:48

they are basically just bits of java code. i think they should translate pretty easily

jtackett21:01:26

oh so you basically make the requests with this library rather than a http-kit for example

codefinger21:01:33

does http-kit obey socksProxyHost system property?

codefinger21:01:44

i think most JVM http clients do

jtackett21:01:54

i don’t know

codefinger21:01:28

otherwise, set a proxy for the client manually. it probably has a way to do that

jtackett21:01:28

that’s what I’m currently using

jtackett21:01:41

what do you mean by that?

jtackett21:01:56

you mean just set up heroku to hit a proxy first then send out the request

jtackett21:01:01

rather than doing it programatically

codefinger21:01:47

no, the addons create a proxy for you

codefinger21:01:07

this example uses apache httpclient (just like cli-http), so you'll need something like this https://github.com/kissaten/proxy-examples/blob/master/src/main/java/Main.java#L103-L110

jtackett21:01:11

I’ll see if clj-http has something similar

codefinger21:01:27

give it a try and ping me if you get stuck. i lurk simple_smile

Oliver George22:01:40

Seems like a nice "better assert" tool

ghadi22:01:41

not sure why new libraries come out that use cljx

macromancer22:01:48

Can anyone tell me a good reason to switch from lein to boot? Or vice versa, perhaps?

macromancer22:01:56

or where I can find a comparison?

Oliver George22:01:02

Three key things I see are: * displaying variable values in assertions, * returning value instead of nil for inline use * dynamic binding to add context to exceptions

Oliver George22:01:34

I'm curious if theres been any thought/discussion about how clojure's assert messages could be improved in these ways.

jaen22:01:22

@macromancer: I'm not sure if I am aware of 1:1 comparison, but this described move from a lein-based project to boot-based project - http://blog.michielborkent.nl/blog/2015/06/06/from-leiningen-to-boot/

macromancer22:01:18

simple_smile looks like i have it bookmarked but have yet to read it.

macromancer22:01:25

@jaen thanks for reminding me!

jaen22:01:04

IMO the biggest advantage boot has is when you're doing Clojurescript - it has more streamlined Clojurescript workflow than lein can offer. It's probably less useful for plain Clojure unless you have specific needs of your build process - with boot you can easily script creating multiple artifacts from the same codebase or somesuch, with lein you would probably have to find/write a plugin.

macromancer22:01:47

i'm doing a lot of clojurescript/figwheel dev, so that sounds pretty good.

macromancer22:01:12

i'm finding lein configuration gets quite large with multiple js targets, though i might be using lein inefficiently..

macromancer22:01:43

what i do want is the composability that boot should offer.

macromancer22:01:18

@jaen cool, thanks for your thoughts

jaen22:01:08

Ugh, I said "less streamlined" when I've meant "more streamlined" xD

macromancer22:01:12

haha, yeah I filtered that out, assuming you had meant something positive.

macromancer22:01:15

This is cool. Clojure executables using shebang notation

jaen22:01:47

Yeah, that is pretty hand indeed, that for simple one-off things you don't even have to create a project, but can load dependencies inline

macromancer22:01:20

This also means I can probably get pretty close to one off clojurescript programs, opening directly in a browser..

jsa-aerial22:01:43

@rfhayashi: conch works great for streaming. use it extensively in streaming pipelines

nonrecursive23:01:06

@macromancer just want to second jaen, with lein it was difficult to do things like also compile&autoreload sass. it looks like there’s a figwheel-like HUD for boot http://deraen.github.io/boot-cljs-tooling/ , haven’t tried it myself though

jaen23:01:54

@nonrecursive: not needed, boot-reload has that hud integrated now

nonrecursive23:01:17

no kidding! I am behind the times

jaen23:01:14

@macromancer: boot-reload + boot-cljs-repl replicate most of the functionality of figwheel; there are some corner cases according to https://github.com/adzerk-oss/boot-reload/issues but I didn't really encounter them in practice so far

jtackett23:01:49

Anyone know the best way to set up a static IP address with Heroku and Clojure

jsa-aerial23:01:31

@joelkuiper: (take-while (complement pos?) [-2 -1 0 1 2 3]) => (-2 -1 0) ;; aka take-until