This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-06-23
Channels
- # admin-announcements (2)
- # arachne (2)
- # beginners (76)
- # boot (241)
- # cider (14)
- # cljsrn (2)
- # clojars (3)
- # clojure (94)
- # clojure-android (12)
- # clojure-dev (33)
- # clojure-gamedev (1)
- # clojure-greece (3)
- # clojure-india (1)
- # clojure-nl (2)
- # clojure-quebec (3)
- # clojure-russia (21)
- # clojure-spec (38)
- # clojure-uk (72)
- # clojurescript (62)
- # cursive (20)
- # datascript (3)
- # datomic (14)
- # devcards (1)
- # dirac (14)
- # emacs (11)
- # hoplon (7)
- # jobs (2)
- # keechma (1)
- # lein-figwheel (9)
- # leiningen (9)
- # luminus (1)
- # off-topic (6)
- # om (13)
- # onyx (30)
- # planck (181)
- # proton (3)
- # re-frame (6)
- # reagent (6)
- # specter (108)
- # spirituality-ethics (7)
- # untangled (3)
@mfikes: i'm getting an kJSTypeObject as the only member of argv in my shellexec handler when invoking the sh
function with a string. When I try to invoke it with an integer or an actual object I correctly get the following error:
cljs.user=> (require '[planck.shell])
nil
cljs.user=> (planck.shell/sh (clj->js {:a "b"}))
In: [0] val: #js {:a "b"} fails spec: :planck.shell/sh-args at: [:cmd] predicate: string?
ERROR
cljs.user=> (planck.shell/sh 9)
In: [0] val: 9 fails spec: :planck.shell/sh-args at: [:cmd] predicate: string?
ERROR
cljs.user=>
any idea why my string arg is being converted into a js object?also any idea how to find out what keys are on that object (which would be my next step for trying to diagnose it myself)?
@johanatan: I'm AFK now but I bet the PLANK_REQUEST
http get stuff (already implemented in planck-c) has some similarities with sh.
and i didn't see any noticeable difference in the way they're setup and the way mine is
i've pushed my code if you'd like to take a look when you have a chance: https://github.com/johanatan/planck/tree/shellExecWorkInProgress
@johanatan: worth looking at the ClojureScript and Objective-C. Perhaps the first arg is passed through.
but given that you said there weren't any changes required to those for the rest of planck-c i figured they would remain constant for this one function as well
@johanatan: I don’t know why clj->js
is being applied to cmd
here https://github.com/mfikes/planck/blob/master/planck-cljs/src/planck/shell.cljs#L39
but that would still produce a string though no?
cljs.user=> (clj->js "blah")
"blah"
@johanatan: cmd
is an array of strings
Actually 🙂 via the spec one or more strings: https://github.com/mfikes/planck/blob/master/planck-cljs/src/planck/shell.cljs#L58
@johanatan: ^ fortunately, half of that code is actually C already 🙂
Hah, the use of C to interoperate with JavaScriptCore traces back to a suggestion of David Nolen’s from a little more than a year ago. I guess he knew one day Planck would be ported to straight C. 🙂 http://blog.fikesfarm.com/posts/2015-05-23-ambly-using-javascriptcore-c-api.html
btw, do you know if there's anything like C++'s std::vector in our C environment here? really not looking forward to a bunch of malloc and free calls to do dynamic allocation
so, this works: https://github.com/johanatan/planck/commit/62a7fb8484edac14840273c5039626da8c4e5456 [wish it were a little tighter but I guess that's C for ya]
Yeah, the last time I was doing malloc
/ free
was in 1994, I think. By the way this will help if you want to check for memory corruption: https://github.com/mfikes/planck/blob/master/planck-c/Makefile#L7 (add the sanitize address flag)
What would we do about supporting :in (and :in-enc) if it turns out file:// isn't supported on C/Linux?
@johanatan: I’m actually not familiar with that particular feature… but I’m guessing we could convert file://
urls to their local path, if it is possible to do so
In the long run, handling of file://
in general might be possible to cope with via the
machinery. (That’s the way http://
can be handled, etc., in a generic fashion.) But yeah, just stripping file://
probably suffices for now.
@mfikes: ok, i think we have a bigger issue with the char encodings. std C doesn't have any notion of these and you'd have to have a bunch of functions like the ones on this page to convert between the encodings (if you wanted to): http://codereview.stackexchange.com/questions/40780/function-to-convert-iso-8859-1-to-utf-8
Yeah… I wouldn’t worry too much about that. I bet most people use UTF-8 if anything and we might be able to take advantage of some stuff that might already exist in JavaScriptCore. For example https://github.com/mfikes/planck/commit/45874f341c02f241f2880b3ae985828f2bbdf99f#diff-b1129fb19d4dc0f5029761d2161e6f22L76
I couldn't find any examples in the codebase or tests that did anything but set string varargs for the 'cmd'
This was my best attempt in the REPL:
cljs.user=> (planck.shell/sh "0" {:env {"blah" "blah"}})
In: [1] val: ({:env {"blah" "blah"}}) fails spec: :planck.shell/sh-args predicate: (cat :cmd (+ string?) :opts (* :planck.shell/sh-opt)), Extra input
ERROR
They are keyword arguments. So
cljs.user=> (planck.shell/sh "env" :env {"foo" "bar"})
{:exit 0, :out "foo=bar\n", :err “"}
What’s exciting these days is you can consider using spec to generate example arguments:
cljs.user=> (s/exercise :planck.shell/sh-args)
([("") {:cmd [""]}]
[("o" "m" :in "9") {:cmd ["o" "m"], :opts [[:in {:key :in, :val "9"}]]}]
[("" "v0" "Cl" :env {}) {:cmd ["" "v0" "Cl"], :opts [[:env {:key :env, :val {}}]]}]
[("Gs" "" :in-enc "RFY" :in "G" :out-enc "3dP")
{:cmd ["Gs" ""], :opts [[:in-enc {:key :in-enc, :val "RFY"}] [:in {:key :in, :val "G"}] [:out-enc {:key :out-enc, :val "3dP"}]]}]
[("O5" "n7j" "d" "" :out-enc "" :out-enc "K0" :dir "a0a")
{:cmd ["O5" "n7j" "d" ""],
:opts [[:out-enc {:key :out-enc, :val ""}] [:out-enc {:key :out-enc, :val "K0"}] [:dir {:key :dir, :val [:string "a0a"]}]]}]
[("cY2" "6Y0" "I8Da" "F" "" "hDrjK" :in "Pb") {:cmd ["cY2" "6Y0" "I8Da" "F" "" "hDrjK"], :opts [[:in {:key :in, :val "Pb"}]]}]
[("q4i" :in "7gDvI7") {:cmd ["q4i"], :opts [[:in {:key :in, :val "7gDvI7"}]]}]
[("G3YDS" "Z2q9" "798rPJT" "8fq" "cN" "u" :env {} :dir "a3oHJ8")
{:cmd ["G3YDS" "Z2q9" "798rPJT" "8fq" "cN" "u"], :opts [[:env {:key :env, :val {}}] [:dir {:key :dir, :val [:string "a3oHJ8"]}]]}]
[("e" "3Cibazb0" "F3q" :env {}) {:cmd ["e" "3Cibazb0" "F3q"], :opts [[:env {:key :env, :val {}}]]}]
[("e44fT" "" "AjhU2X2XC" "8jVEyD3" "SgHp" "R6lKza78P" "TtFDVMdVz" "Se5B53Cdh" "F948Sdg5")
{:cmd ["e44fT" "" "AjhU2X2XC" "8jVEyD3" "SgHp" "R6lKza78P" "TtFDVMdVz" "Se5B53Cdh" "F948Sdg5"]}])
Of course, the spec hasn’t been told that, for example that ”9”
is not a legal string encoding 🙂
@johanatan: By the way, in case it helps, planck.shell/sh
is an imitation of clojure.java.shell/sh
https://clojuredocs.org/clojure.java.shell/sh
Ahh, nice. I had actually tried a variation of that with :cmd before the first string arg
Any idea on this one? Looks like the value I'm specifying for :dir is being sent through as a vector which as-file
doesn't have a protocol method define for.
cljs.user=> (planck.shell/sh "0" :env {"zblahbb" "blahzz" "abc" "def"} :dir "/home/jonathan")
No protocol method Coercions.as-file defined for type cljs.core/PersistentVector: [:string "/home/jonathan"]
cljs.core/missing-protocol (cljs/core.cljs:266:4)
(planck/io.cljs:259:33)
planck.shell/sh (planck/shell.cljs:49:60)
doh! same error happens when the directory actually exists:
cljs.user=> (planck.shell/sh "0" :env {"zblahbb" "blahzz" "abc" "def"} :dir "/Users/jonathan")
No protocol method Coercions.as-file defined for type cljs.core/PersistentVector: [:string "/Users/jonathan"]
cljs.core/missing-protocol (cljs/core.cljs:266:4)
(planck/io.cljs:259:33)
planck.shell/sh (planck/shell.cljs:49:60)
@johanatan: there is an error in the spec handling here...
(s/conform ::sh-args [ "0" :env {"zblahbb" "blahzz" "abc" "def"} :dir "/Users/erik"])
{:cmd ["0"], :opts [[:env {:key :env, :val {"zblahbb" "blahzz", "abc" "def"}}] [:dir {:key :dir, :val [:string "/Users/erik"]}]]}
@mfikes: after taking your latest commits, I'm seeing the following error:
Jonathans-MacBook-Pro:planck-c jonathan$ ./planck
Planck 2.0
WARN: no bundled sources, need to run script/bundle-c
ClojureScript (Unknown)
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Exit: Control+D or :cljs/quit or exit or quit
Results: Stored in vars *1, *2, *3, an exception in *e
WARN: no bundled sources, need to run script/bundle-c
The goog base JavaScript text could not be loaded
Do make bundle-and-build
in the planck-c
at least once to get it bundle things up. The bundle-c
script is off in another part of the tree but the makefile knows where to run it
(You can rapidly iterate on the native part, with bundle.c
containing all of the AOT-compiled JavaScript.)
yea, that's what i've been doing and so forgot about the initial make bundle-and-build
i did check that bundle.h/c were present though and they were-- i suppose they were perhaps stale due to the code changes though
Don’t understand… unless you are saying that the command is just the initial segment of the argument list that is strings, until you hit the first non-string.
spec’s regular expression machinery (`cat`, et. al.) is awesome for expressing these concepts
The high level idea is that a structure can be a sequence or a map. cat
is perhaps the top-level for the first, while keys
is the top-level for the other. (probably not 100% accurate), but fits my mental model for now.
one question: should it be safe to take a c-string obtained from value_to_c_string
and do an unbounded strlen
or other str
operation on it? I am imagining so as I can't see how one could get an unbounded string from ClojureScript through the JS machinery and into these native bits
i.e., it should be safe to assume that the strings obtained in this manner are null-terminated
one other thing: can you explain what self.context
is/where it comes from and how it is different than ctx
(if it is actually different)? I don't think I have anything analogous to that in the native function:
https://github.com/mfikes/planck/blob/7b8fb645177a5828de87b51cca9ab883402e9436/planck/PLKClojureScriptEngine.m#L638-L640
would it be ok to just create those return values on the ctx
that is passed into the function?
@johanatan: It should be safe to assume that C strings obtained from that function are NULL-terminated.
and for my second question, i just went ahead and tried it with ctx
and it seems to work
@johanatan: self.context
and ctx
would be the same thing
in that case, i'd say let's go ahead and get a PR in for this as it would be super useful even without :in support
It may even get us to the point where we can run the existing test infrastructure which uses sh
@johanatan: Looks like you may have inadvertently left some debug console printing
OK… that’s likely to cause it to not be usable when running the existing integration tests
@johanatan: Cool. Perhaps after that is in, I can make some new Linux binaries and put them on the Planck alpha download page 🙂
@johanatan: Yeah, I merged prior to your latest push. (Sigh, mutable PRs.)
[Sorry about the race condition-- i figured i could get the fix in before you merged the PR]
But… if you are asking about the full Planck test suite… not yet… there are other things failing stil.
Hmm, I don't see any tests in this one-- only the build itself: https://clojurians.slack.com/archives/planck/p1466707684000309
Right, so when I asked about the tests, I meant the ones that you said might run with the impl of sh
with debugging removed.
@johanatan: I was referring to the stuff exercised by script/test
(and script/test-c
)
script/test-c
doesn't fully work yet given the gaps that Planck 2.0 has relative to 1.x
Part of Planck's tests originate from prior to cljs.test
being functional under bootstrap, and involve crude comparisons of stdout with what is expected. (So writing debug output would cause those crude tests to fail.)
And, yes, parts of the test infrastructure use Planck itself to run scripts. Your changes will help Planck fundamentally be able to execute all of that on Linux.
So, next step for me will be to get async version of sh going. Do you have any guidance on how to accomplish? Should I add a dependency on core.async and add a new function: sh-async
which returns a channel which the {:out ... :err ... :status ..} payload is sent over when available?
@johanatan: I wouldn't force people to take on a core.async
dep, but instead support a callback. A callback can be converted to core.async
by users interested in it. This is also good post in the subject: http://www.lispcast.com/core-async-code-style
Ah, yea, I remember that. As long as the callback is the last argument or something, there's an automatic conversion?
are you ok with the name sh-async
or is there a common naming pattern for that sort of thing?
There is no automatic conversion, but it is easy for a client to pass a in lambda that puts a result on a channel.
In terms of API, I'd take a look at the http and other APIs in Planck to get a sense. (I suspect a new arity won't work, and http and sh have different ways of passing optional stuff, so maybe -async
might feel right in the end.) might be worth experimenting
@johanatan: I'd suggest looking at cljs.js
. It went with cb
as last argument
@mfikes: pretty sure there is an automatic conversion provided by promesa or one of the libs I used recently
oh, no. actually it was <<<
from here: http://www.lispcast.com/core-async-code-style
Seems like accepting a cb
then allows for any preferred style to be used on top of it.
I'm thinking another wrapper that does:
(sh-async other-args :cb callback)
might work
It is tempting to revise sh
to return nil
if :cb
is present. Wonder if that would be natural or a hack