Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2017-04-05

Channels

#onyx
<
2017-04-05
>
devth01:04:00

i'm getting an error during lein test in CI, which runs in a docker container that I don't have a ton of control over: 

java.io.IOException: No space left on device
clojure.lang.ExceptionInfo: Error in component :messaging-group in system onyx.system.OnyxPeerGroup calling #'com.stuartsierra.component/start
     component: #<Aeron Peer Group>
i think the solution is typically to increase shm-size but I don't have the option to do that in this env. i also can't mount volumes. any other ideas/configuration to workaround?

devth01:04:08

there isn't an alternative :onyx.messaging/impl for test i could use is there?

gardnervickers01:04:44

@devth you can mount a drive at /dev/shm

gardnervickers01:04:56

Oh shoot misread that

devth01:04:00

yeah, don't have the option in this case 😞

devth01:04:12

using gitlab CI

devth01:04:24

they don't expose volumes or settings for their runners yet

devth01:04:49

now i'm exploring running mount -t tmpfs -o remount,rw,nosuid,nodev,noexec,relatime,size=256M tmpfs /dev/shm inside the docker container on startup

devth01:04:56

expecting it to fail though

gardnervickers01:04:02

:onyx.messaging/allow-short-circuit? true

devth13:04:22

tried with this but it still produced the same "No space left" error. changing the aeron.dir fixed it though. thanks!

gardnervickers01:04:50

That skips Aeron and uses core-async for message transport. I'm assuming you're running everything in the same process for your tests?

devth01:04:01

thanks. trying that

gardnervickers01:04:45

@devth That was deprecated in 0.10 however

devth01:04:21

ok. we're still on 0.9.10 for now

gardnervickers01:04:40

Does GitLab let you allocate more memory? If so you should be able to mount like 512mb or so at /dev/shm if you can run your containers in privileged mode.

devth12:04:06

nope. as of now there's almost nothing you can configure in the Kubernetes executor for gitlab. it's pretty new. i requested the feature to specify stuff like this.

lucasbradstreet02:04:09

Aeron will let you move the dir location

devth13:04:24

awesome, this solved it! thanks.

lellis20:04:15

Hi! Im trying to put headers in a http-output plugin like this 

{:url  :headers {X-Amz-Date 20170405T204406Z, Authorization AWS4-HMAC-SHA256 Credential=MYCREDENTIALS, SignedHeaders=host;x-amz-date, Signature=MYSIGNATURE}}
but the request dosen't have these headers. Is there something more to do to send more headers in a http-output out?

lucasbradstreet21:04:48

@lellis I think the headers need to be under :args

lellis21:04:02

@lucasbradstreet Yeah! Success! By the way when we set args the request method turn to POST.

devth23:04:56

i'm working on securing my infra and realized ZooKeeper does not support server-to-server SSL. does onyx provide any mechanism for encryption before data is sent over the wire?