Fork me on GitHub
#beginners
<
2016-02-13
>
Drew Verlee19:02:05

Use #C06DT2YSY instead of #onnext I think. @adampash

akiva20:02:47

Even David’s not in #C0DT1AZD3. We really, really, REALLY need to stop the proliferation of subchannels. This isn’t NTTP circa 1993.

coyotespike20:02:22

So, I'm still working on a scraper that will work on secured pages. Using @donaldball's helpful advice, I figured out how to (a) get the CSRF-token as well as the cookies and (b) POST those and my creds and log in. I can tell I'm logged in because it returns 302. But, when I try to use clj-http cookie store to GET the secured pages, it still won't let me.

coyotespike20:02:35

Any ideas greatly appreciated, I feel like I'm pretty close here.

akiva20:02:59

I don’t think I can help in this particular situation but if you have this code checked into Github or Bitbucket, it might give the context someone else could use to be of help.

doddenino20:02:12

I'm using phantomjs in a leiningen project, and I have to pass the path to the executable with :jvm-opts ["-Dphantomjs.binary.path=..."]. Is there a way to set this with an environment variable so that the path is not hardcoded in my project.clj? Thanks!

jonahbenton20:02:14

hey @coyotespike, yes, or if you can share the code snippet to make the GET call, someone will be able to help...

coyotespike20:02:58

@akiva @jonahbenton 😎 I'll prettify the code snippet a little and throw it in a gist

jonahbenton20:02:27

hey @doddenino: couple of thoughts: * the app can just use (System/getenv) and (System/setProperty) directly. * you can populate a JVM_OPTS environment variable, and lein will pull those in (see bottom of https://github.com/technomancy/leiningen/blob/master/sample.project.clj) You can also do uglier things like have a system property that tells the app which environment variable to pull in with (System/getenv)

coyotespike20:02:53

Yoicks! I just got it!

coyotespike21:02:04

For the GET request to succeed, it has to use the same cookie store that the log-in POST has just updated. I think I wasn't doing that right. Pulling it out of the let-statement and into a variable, like (def cookie-store (cookies/cookie-store)), fixed me up. Thanks for the prompt to refactor, guys 😉

coyotespike21:02:40

lol yeah, I used to have a literal rubber duck - maybe I need to get him back simple_smile

doddenino21:02:08

cool, thanks! I'll look into that