Flex disconnected from the oven feedthrough pins and replace by an adjustment pot. The flex did not improve during desoldering, but was knackered already, so...
Very interesting phenomenon today: upon powering up the oven heater started
and regulation kicked in (as designed, I assume). That was a one-time
effect though. In other words: intermittent problem. Great. Not.
Based on some consideration, and while muttering the famous "Who Dares Wins" after
maesurng things in the thermistor circuit I decided that the thermistor
flex wrapped around the crystal cavity must be flakey.
Flex disconnected from the oven feedthrough pins and replace by an
adjustment pot. The flex did not improve during desoldering, but was
knackered already, so...
That proved interesting. Bottom DMM shows the divided-by-two reference
voltage generated within the oven. Top DMM shows the voltage from the
thermistors, or rather: from the adjustment pot in this case. And the
heater draws considerable power.
Long story short: I cut the flex wire connecting the thermistors to the
feed-through pins and replaced them with some Tefzel insulated wire wrap
wire. I guess Rick might take a dim view of this but lacking an original
replacement flex you have to make do with what you have available...
Test setup running now the flex has been bodged..
During warmup the output voltage on the thermistor bridge slowly drops (top
DMM). The reference voltage (2.5V divided by two, and produced inside the
oven) shows no major change (bottom DMM). Note the 100.93 degrees
centigrade on the oven label.
There are now a green and a yellow LED blinking, the red LED is no longer
lit. Appears to me like a good sign.
Shown on the left the original parameters set on the main PCB's PIC. On the right
the monitoring of the oven control loop.
Remaining questions/actions
Lacking any docs, it took a bit of tinkering to get NGOCOMM talking to the
E1938A. Trick is to first do a Sync PIC and subsequently a Get RED. This fills the NGOCOMM display
with the parameters of the PIC controller.
I guessed selecting the Ramping option should allow me to get heater
control. Alas, nothing visible happened, no heating current was drawn.
Far more interesting was opening the control loop using the Loop Control Opn
button, followed by a Send RED.
Immediately the heaters started drawing about 3.3 A at 5V. Closing the
control loop with the oven hot immediately disabled the heaters.
The Loop Status window shows various measurements. With the Cnv checkbox
checked the hexadecimal values are translated into something more human
readable.
Observations sofar (unordered)
Open questions
Based on Bob Camp's questions on the current drawn by the heaters I have again pulled the 3 power FETs and measured things in a somewhat structured fashion (I hope..).
For the 'hot' measurements I allowed the oven enough time to reach
10MHz nominal, under the assumption that that is a reasonable indicator
that the oven temperature has reached the correct value.
For the
WARM_UP_HTR that took approximately 25 minutes, for the RIM_HTR and
MASS_LID_HTR this took approximately 10 minutes.
In-between measurements I cooled the oven back to room temperature (~ 24 degrees C) using a fan.
The table below shows the outcome of it all. Measurements with respect to ACOM.
| Heater | FET | Current cold | Current hot | Gate voltage cold | Gate voltage hot |
|---|---|---|---|---|---|
| WARM_UP_HTR | Q8 | 0.88A | 0.75A | - | - |
| MASS_LID_HTR | Q9 | 2.24A | 1.90A | -5.09V | -4.88V |
| RIM_HTR | Q10 | 1.24A | 1.05A | -5.07V | -4.84V |
So, yes, the current drawn by the various heaters indeed drops when the oven heats up. Tempco of the heaters themselves I guess.
Based on the change in gate voltage it appears to me that the temperature sensing is working, the digital PI^2D is doing its thing etc. At least to a certain extent.
Again I noticed that ever so often on powerup the E1938A starts up drawing ~3A for a couple of seconds and then falls back to 60mA. For total lack of better ideas I get the feeling the PIC chip senses something on powerup that it does not like and shuts down all heating. With the FETs out, the DAC drives the gates of Q9 and Q10 based on the temperature it measures???? (lots of ?).
With the oven hot, I checked the VREF_2.5 on the DB25 connector. Initially
what I measured with respect to VREFCOM did not make sense. Measured
with respect to AGND it was 2.538V Not good, as it is
not "the HP way" to be off like that. Some close inspection later:
on my unit R200, a 0 Ohm SMD resistor was not populated. R200
should be at the empty pads just above the big DALE resistors, to the left of
the DB15 connector. Once a fresh R200 had been installed, I had a nice 2.5018V for VREF_2.5
with respecto to the now proper VREFCOM.
All in all: lots of entertainment, but still no clue what is going on. My current gut feeling tells me "something fail safe kicking in in the PIC firmware". But why, what is causing it? Must be someting in the oven itself, based on what Morris wrote earlier.
I spent a bit more time on the E1938A. The "Hmmm" part in the June 6 write-up is (of course, in retrospect) going nowhere. I assumed I had isolated the thermistors from the rest of the circuit, and of course I had not. Duh..
Today I checked all supply voltages, they appear to be what they should be, so +5VD, +5VA etc. The VREF2.5 originating from within the oven is also reasonably OK at 2.534V. As the oven is not currently heating up, I assume the voltage reference will also be working properly when at the nominal ~100 degrC setpoint.
During testing I noticed that sometimes the E1938A 'boots' up drawing ~3.5A from the 5V rail. That is only for a few seconds, dropping back to say 50-70mA. More often than not it does not start with the high-current surge, instead just drawing its usual 50-70mA directly from power-up. I have yet to find any pattern in this behaviour, as I suspect that solving this will be the fix for the entire 'oven does not heat-up' issue.
Surgery in progress.
For good measure, I pulled the 3 power FETs that drive the 3 different heaters for inspection.
The PCB material shows signs of heat stress under Q9 and Q10 (the ones
with heatsink). Below the heatsinks there was some environment pollutant,
now cleaned off. The FETs were identified as FETs by my little Chinese
gizmo/component tester so I assume they are ok.
With the FETs out of circuit I could test the actual heaters by simply
connecting the drain-source on the PCB (read: the tweezers on the photo is
the D-S short). All three heaters, so WARM_UP_HTR,
MASS_LID_HTR and RIM_HTR work. During warmup the current gradually reduces
from ~3.5A to ~0.8A. All of this is good, proves all flex
wires/heaters work. No shorts to ground, no unsound drawing of power.
Flexes are unrepairable (by me) in case there had been a failure.
I noticed a CURR_LIM section on the schematic. Wondering if that was maybe kicking in inadvertently, I pulled R124 to see if that made a difference. It did not.
Allowing the oven to get really hot (I suspect
around the nominal 100 degr C, do not currently have a contact thermometer
available) makes the oscillator go to 10.000000 MHz.
Likely good news I think. Get the oven fixed and the frequency will also be OK
again.
What I also did: connect an FTDI TTL level serial-USB dongle to J101, the
serial port. I think it is 9600 baud (not certain). Whatever it emits is,
based on what I saw up til now, not intended for human consumption.
Reading the Development Paper I understand the serial can be used to set
oven control parameters etc. Maybe (I hope...) also read some status on what is
going on. In any case I need more information on what it is the serial
port can do, and maybe dedicated software to actually use it?
Also useful would be the meaning of the 4 LEDs DS4, DS5, DS6 & DS7. DS7 is blinking, looks like that syncs with what is emitted on the serial port. A heartbeat indicator maybe?
Any information is more than welcome! Suggestions too of course.
Based on that I used my stereo microscope to take a closer look at the inside of the oven, especially at the 3 thermistors and the flex that interconnects them to the PCB inside the oven. Unfortunately my microscope does not have a camera mount, so please bear with me for the pictures made using the microscope...
Context for the next set of pictures. And the header pins/flex connection
for the thermistors. The flexwire definitely has degraded over time.
The flex, once carefully bent, shows more of the cracking :-(
Adjacent a close up of thermistor #1. The thermistors are mounted using a
flexible polymer, my guess is that this is a thermally conductive silicone compound.
Thermistors #2 and #3 in close up.
Now the interesting part: thermistor #1 measures at 20.09 kOhm, #2 at 20.37 kOhm, #3 at 19.93 kOhm.
The corresponding section of the schematic is shown below.
Measuring the resistance on the header pins of the thermistor flex (PIN 1 and PIN2 at the right hand side of the schematic) shows 14.46 kOhm. Measuring on the DB15 connector Pins 2 and Pin8 (DB15 interconnects the oven to the mainboard) also shows ~14.46 kOhm.
Hmmmmmmm.....
Interestingly enough, after the first powerup and after cool down the initial 3.5A current did not re-occur. Which is unexpected.
Reset button and some status LEDs, some blinking. Unfortunately no
information found yet on what each of them signifies.
The flex PCB contains the heating wiring. Note the 3 Torx screws, sealed
in some lacquer, to ensure the hermetic sealing that is referenced in the
design Papers. Again a label with the temperature setting. Side note for
the SI folks: 100 degr C aka 373.15 K is the temperature of boiling water. So
when operational this oven is truly hot...
The actual OCXO stainless stell 'hockey puck' removed from the clam shell. Again a sticker
with the temperature this particular crystal requires for optimal stability.
To the left side 6 feed-through pins that connected the flexwire/DB15
connector to the electronics PCB inside the oven. Two more feed-hrough
pins on the right side.
The sealing lacquer proved to the easily removable. It is very hard and brittle,
it chipped off after some gently prying. Damaging the heater flex would be
bad news indeed... Once the lacquer was removed, unscrewing the Torx
screws was uneventful. Hurrah!
Lacquer chips and the inside of the lid of the stainless stell hockey puck.
The flex to the left connects to the heater. The yellow discolouring on
the stainless steel lid puzzles me, maybe out-gassing of plastics inside
the hot oven set onto the lid?
"We're in like Flynn", to paraphrase 'Dave from Australia' (yes, EEVblog).
All in all far easier than I had feared after reading the "hermetically
sealed" story in the research & development Papers. In fact, I had dark
suspicions of a "not repairable" unit. Quite a relief therefore.
A truly elegant design! The green blobs on the centre ring surrounding the
crystal are the 3 thermistors. They are connected to the electronics module
via a small, 2 pin flex/header pin connector.
Flex PCB connecting to the crystal, with what looks like a trimmer
capacitor to the left. The green blobs is glue fixing the thermistors to
the crystal enclosure.
Another flex PCB is glued to the outside of the crystal enclosure,
serving as the interconnect of the three thermistors. Each thermistor
measured at about 18.8 kOhm. The resistance on the 2 header pins on the
flex is about 14.5 kOhm.
All thermistors have their own portrait. Added numbers to identify one thermistor
from the other.
The O-ring mentioned in the development Paper. It is totally shot, it
breaks if you tap it on the table. Most likely degeneration of the O-ring's
polymer due to heat.
Note the temperature, 103.70 degrC on the label.
Detail pictures of the components used. Not sure what to think of the
"100 C -560 pF" marking.
Detail pictures of the components used.
Detail pictures of the components used. Quite a bit of flux residu on
the hand soldered components.
Detail pictures of the components used.