Remote Debugging Recently almost all ethernet devices are manufactured as LOMs such that it's getting harder to buy stand-alone PCI/PCIe based ethernet controllers in market. Buying a new motherboard is not an option to me because I can't afford to do that to debug driver issues. Some controllers are only available on mobile devices and it's hardly possible to get such mobile controllers without buying laptops. The same is true for controllers targeted to high-end server market. I can't get buy expensive 1U server supporting fiber links. In these cases it may help a lot to debug driver issues if I manage to get a remote access to the box. I normally do not request remote debugging mainly because it consumes a lot of time and require two systems with appropriate setup. In addition, some issues couldn't be debugged with remote debugging session and some cases it require remote power circuit to restart the target system. Given that user can reset or restart target machine when I encounter stuck condition it's possible to debug a lot of issues with minimal setup. Here is a minimum system/network configuration for me to get remote debugging working. Please read entire contents carefully and send remote debugging request mail to yongari@FreeBSD.org If I find some spare time I may be available to accept the offer. If you have any questions feel free to contact me. 1. No production server Most debugging works are destructive for normal operation of service so the target system should not be a production system. If you have spare identical system, use that system instead. 2. Two systems connected over serial console/network Basically the remote system that acts as a controlling host should have two or more network interfaces to debug a ethernet driver. The target host which is suffering from a driver issue would be connected through both serial and ethernet. The controlling host and target system should use different ethernet driver in order to isolate issues to a single ethernet driver. The remote system should be connected with serial cable to gain full control for target system in case of network failure. In order to monitor activity of patched driver on a target system I need serial console attached to the target system. Having a serial console on controlling host will help a lot. Note, I don't put much trust on USB serial adapters so if you have dummy serial cable please use that instead of USB serial adapters. 3. System/network configuration Here is a minimal network configuration for remote debugging. Internet Host A serial cable Host B <-----------> [ NIC0 serial port ] <------------------> [ serial port ] [ ] [ ] [ NIC1 ] <------------------> [ NIC0 ] private network(e.g. 10.10.x) [optional extra NIC] <------------------> [optional extra NIC] optional private network(e.g 192.168.x) Host B is the system that has network driver issues. Host A would be used as controlling host as well as accepting incoming connection from me. I don't need to access your internal network so it's sufficient to set up a private network between host A and host B. The serial cable will give me complete control for host B. I don't need Internet access from host A and host B but it would help a lot if there is additional optional network between host A and host B. If I have a large patch file on host A, it's hard to get it the patch on host B without extra network given that NIC0 of host B does not work at all. In that case, I have to use copy/paste method through serial cable. This is error prone and takes time so if you have additional ethernet controllers, please setup additional network between host A and host B. Note, Host A should be able to accept remote SSH connection from me. Setting up a remote access GUI tool like TeamViewer on Host A is highly discouraged due to lack of file transfer feature to target system. Also these kind of GUI tools are too slow to work on remote location unless you have high speed network. 4. Kernel source code and build environments I'll have to experiment various things to narrow down the issue on target system. This involves patching/rebuilding kernel so kernel sources and compiler should be installed on target host B. 5. Root/user account access To log in to host A I need a normal user account on host A. For host B, I need a root account as well as an user account to rebuild kernel and reboot system. I do not need a root account on host A though. But it would be better to have root right on host A to get the output of tcpdump or other network activities. 6. Required software To test how well patched drivers work, I need netperf benchmark program(/usr/ports/benchmarks/netperf) installed on both host A and host B. This program would be used to check performance and stability of the ethernet driver. 7. Security Because you have to accept incoming SSH connection from me you have to allow SSH connection initiated from me. I can give you a SSH public key and an IP address which I'll use during remote debugging session such that you can configure your host A to accept connections initiated from me. Also you may want to configure your firewall rule such that host A and host B could be on a confined network in your network environments. Once I receive a mail from you that says all configuration(system, account information, network topology etc) was done on remote side, I'll send you a mail before logging in to the remote system. Debugging time may vary but normally it would take two or three days but it wouldn't last for more than a week. If I couldn't find a clue or fix for the issue within a week I'll give up remote debugging session. However it's also possible for me to request another debugging session in future. After either I've done or gave up the debugging I'll send you another mail that explains current status and what I tried and what you would want to do after receiving the mail(e.g. closing account, disabling remote access etc). If you're concerned about what I did in debugging session, you can wipe out all contents in disk and reinstall operating system. 8. NDA/Contract If you need a NDA signing from me to allow remote debugging, please let me know that first. I want to have chance to review NDA first. Normally I don't sign any NDAs but there could be some exceptions on some sites where all access should be granted by NDA. If the NDA has too severe conditions/restrictions, I may reject to sign on it. 9. Liability You should be aware of possible breakage/damage of controller or data loss on target system. I don't think executing some debugging code can cause severe system damage or data loss but if that thing happens or if you think you encountered it after the debugging session, please be aware, I have no liability to indemnify you for the damage. If you can not agree on this condition, do not request remote debugging session. Thanks.