| Index: trunk/udplog/TODO |
| — | — | @@ -0,0 +1,38 @@ |
| | 2 | +-------- Original Message -------- |
| | 3 | +Subject: udp2log status and future |
| | 4 | +Date: Tue, 08 Nov 2011 10:58:19 +1100 |
| | 5 | +From: Tim Starling <tstarling@wikimedia.org> |
| | 6 | + |
| | 7 | +I've wrapped up my udp2log work for now. The idea of using |
| | 8 | +non-blocking writes to pipes didn't work out as I had expected, so I |
| | 9 | +made the blocking mode the default, as it was previously. |
| | 10 | + |
| | 11 | +The idea was to do non-blocking writes with only a 64KB buffer in |
| | 12 | +between udp2log and the pipe, and to drop blocks if any write fails. |
| | 13 | +It turns out that the buffer overflows very often, leading to a loss |
| | 14 | +rate of around 10% even with low CPU load. I'm not exactly sure why it |
| | 15 | +overflows, but any stall of the pipe process on the order of 1ms would |
| | 16 | +cause this. |
| | 17 | + |
| | 18 | +To fix it, I would need to do a proper job of the async writer. It's |
| | 19 | +not clear whether the half-async design, with blocking UDP reads and |
| | 20 | +non-blocking pipe writes, would work correctly with larger buffers. |
| | 21 | +It's possible that the write side would be starved of event response |
| | 22 | +opportunities while the read side blocks. |
| | 23 | + |
| | 24 | +Reads are done in very small (~1400 byte) blocks, because that's how |
| | 25 | +big the packets are. If the read side were non-blocking, there would |
| | 26 | +need to be an epoll_wait() call in between each recv() call, doubling |
| | 27 | +the number of syscalls. Testing indicates that this also doubles the |
| | 28 | +amount of CPU time required for the read side, which would be a |
| | 29 | +serious performance issue. So my idea is to split the read side and |
| | 30 | +the write side across a thread or process boundary, using the existing |
| | 31 | +buffer pipe to communicate between the two processes. |
| | 32 | + |
| | 33 | +The read side would be blocking, and would just aggregate UDP packets |
| | 34 | +into 64KB blocks and write them to the buffer pipe. The write side |
| | 35 | +would be a typical async event-driven process, with a function |
| | 36 | +basically the same as the current Udp2LogConfig::ProcessBlock() except |
| | 37 | +that the backlog buffer would be a std::deque of Block objects instead |
| | 38 | +of a single Block object. |
| | 39 | + |