Repairing a CR-78

This is a post that's been in my backlog for a while, because I knew it would take some time to write it...

Back in February this year (2013) I got a request from someone if I could repair a CR-78 with severe battery leakage. A repair-shop that was asked about it wouldn't do it because it would take too long and not be worth the time/money.
Well... I'm cheap, but I leave no guarantees that I'll fix it 100% or that I'll do it quickly, but I'll do my best. That was good enough for this guy, so he let me have a go at it.
The actual "practical" symptoms weren't actually that bad though:  Not possible to save rhythms

However, considering the massive leak, it was clear that this machine needed more than just a quick repair.
No way should anybody give up on a such a fine machine unless it's totally hopeless.
Challenge accepted!

Evaluating the damage

When opening it up and inspecting the PCB, I wasn't sure what to expect. (All images are clickable)

Right away it was clear that there was corrosion on the actual case and that half of the logic PCB had an overall white tint to it, just like the one you see near leaking batteries. The original battery had been removed before the unit was handed to me, but it wasn't hard to see where it had been. I looked at the analog board (not seen here, it's underneath the logic PCB on the other side of the mounting plate), but it looked really clean and in good condition. Lucky! I decided not to mess with it at all.

Looking closer at the logic PCB I saw very badly corroded traces and components. Here are some of the worst parts I could see.

The instrument trigger transistors that sit next to the battery

Diodes for the instrument selector switch

EPROM and RAM sockets+chips with decoupling capacitors. Battery pad bottom left.

Opposite edge of the PCB!

As you can see, it was bad, very bad. Almost half the PCB looked like this more or less. The half of the PCB where that battery didn't sit looked a lot better. However, as you can see in the last picture, traces and components near the opposite edge were also damaged. I assume this is caused by the corrosive gas leaking from the battery and gathering at the other end of the board, so I assume the CR-78 had been resting on its side.

What do you do when you see this? Just clean it off? No.
There was only one right thing to do:
Remove everything in the affected area, clean the PCB properly and replace ALL the components in that area with new ones, except parts that are extremely hard to find replacements for.

More bad news

I was informed that the owner was told to put bicarbonate on the area where the battery had been, to neutralize the battery acid, because bicarbonate is a base. When I first opened the unit up, I had to clean that off. If anyone reads this, please make note:
NICKEL CADMIUM CELLS ARE NOT ACIDIC! THEY HAVE AN ALKALINE ELECTROLYTE!
When people think of battery electrolyte they probably think of car batteries (i.e. lead and hydrochloric acid). NiCad batteries are not constructed the same way. The electrolyte is Potassium hydroxide, commonly called caustic potash, and it's an alkaline/base, so you can't neutralize it with another base.
It doesn't help. If anything, you're pouring more gas on the fire by doing it.
But, since it's just a dry powder, it's really not such a big deal and I am pretty sure it had zero effect during the time it had been left there before I got it.

If you want to remove alkaline electrolyte leaks, I would suggest using either just water to wash it off, or something mildly acidic first, and then water, and finally alcohol. Make sure you dry it properly!
At least, that is my opinion. Correct me if I'm wrong...

Anyway, it wasn't really bad news, just a misconception that deserves some attention.

The really bad news is this:

The battery in the older versions (serial numbers up to 862899) used a 5.6V 4N-100AA NiCad battery. The "newer" ones (serials 872900 and up) used a lower voltage 3.6V N-SB3 NiCad battery.

Here is a quote from the service manual under the comment about the 5.6V battery:

1. D109 is removed at the factory to increase the charging current. However, there are some products having D109 on the market. REMOVE D109 on the first ocassion.
2. (after D109 removed) Never turn on the power switch with 4N-100AA DISCONNECTED. HIGHER voltage will ruin IC102 and IC103.

IC102 and IC103 are the uPD 5101LC RAM ICs. In a way, the battery, charged by the +15V rail, acts as kind of a voltage reference for the RAM. If you turn the thing on without the battery in place, the RAM receives way too high voltage. They're designed to work within 2V-5.5V.

Why am I telling you this? ... You guessed it.

This unit had a serial number lower than 872900.
D109 was missing.
The battery was removed.
The machine was tested after the battery had been removed.

I don't blame him for it though. It's a stupid design. Unless you read that very page in the service manual you have no idea about this problem and I don't think anybody expects it. I sure haven't seen it in any other machines and I didn't know about it until I read it myself.
The RAM isn't just any standard RAM either. It's a staggering 1024 BIT (256x4) SRAM, 22-pin, 400mil/0.4" pitch (!) chip.
Luckily I located some on eBay and ordered 2 before testing, as I assumed they were toast.

I already blogged about the battery replacement procedure separately. If you're only interested in that, you can read about it here: http://synthpeter.blogspot.se/2013/07/roland-cr-78-non-rechargeable-lithum.html

Finally, a major annoyance was that there were a lot of ribbon cables coming from the front panel and the analog board, and they were soldered, and then glued, directly to the logic board.

I'll just remove these and solder them back temporarily when I want to test it... I don't think so!

There were more than those, but that was the messiest area on the board.
I guess they never expected anyone to ever want to service these things. If I was going to be able to test this, I had to put connectors for every ribbon cable.

Removing everything

It was time. I started labeling the ribbon cables and took photos so I knew where and how they had been attached. I took photos of every small group of wires, resistors, capacitors, diodes, ICs and sockets, documenting their position, orientation, designation and value.

I can write pages and pages about this, but I'll just put a lot of commented random photos from the disassembly process here for your enjoyment :)

A RAM IC

Label everything!

Board removed!

Here it is! Time to start removing components.

You can see where the infamous D109 was located (betwen the diode and 120ohm resistor).

Area where the battery and lots of ribbon cables were attached.

Area where trigger transistors and resistors were. Nasty!

The instrument selector diodes were here.

RAM and EPROM sockets. Yes, these definitely have to go...

...and it looked like this below the EPROM socket...

...and like this under the RAM sockets.

Most of the area under the CPU socket was also damaged.

Some stuff near the opposite edge of the board from the battery.

Document the value and orientation of everything. Also opposite end of the board.

And finally...

Everything I planned to remove is removed. Note I have already started rubbing some crust off near the battery.

Cleaning time

Finally it was time to remove as much of the crust and as possible. I started by using a small hobby drill with polishing accessories, but it was very difficult to get it right, and if you touched some other part it had a tendency to just fly jump around and scratch everything. I also tried scratching some damaged pads and traces with an x-acto knife, but this was sooo slow and difficult.
I wanted a better way to cover a large area quickly. I found a ball of old steel wool in my toolbox and tried it. Worked pretty well, but I wanted something even better.
After a while I came up with a more efficient way of doing it:
Sandpaper! :)

I never thought I'd use one on a PCB, but this one has really thick traces and there was a lot of crust and crud all over it, so I gave it a go. It actually worked surprisingly well! See for yourselves :)

The trigger transistors and resistors area. Much better!

The EPROM socket pads. Quite an improvement!

OK, sanding scratches the PCB itself, but who cares. It's not that bad, and it's a small and insignificant price to pay compared to the speedy results.

Aciiiiid! I mean.. Base!

This was the real mechanical removal process, but I was still worried there were electrolyte leftovers under components that I had chosen not to remove and in general all over the board. An acid would be a good way to neutralize any active remaining electrolyte and I read that vinegar might do the trick.
So, I went and bought a bottle of 24% vinegar and basically wet the entire board (I didn't drown it completely, with switches and all, only the board itself and the component pins) and let it soak for a few minutes. Now I don't want to cause any damage, since acid does eat away on healthy metal as well. I just wanted a neutralizing effect on any remaining electrolyte, so I didn't leave it for too long.
After that, I washed it all off using regular tap water, which is almost neutral (at least compared to the vinegar and electrolyte). Finally, I gave it a good wash using a lot of isopropyl alcohol.

I did have to pay special attention to the pads the battery had been soldered to. They were blocked and full of hardened corroded solder. I applied the vinegar to the pads and let it react a bit. I actually saw bubbles forming in it! So yes, something was definitely happening there. After waiting for a few minutes I cleaned it off using water and isopropyl alcohol. I then re-heated the pads and applied new solder to them and used my solder sucker to remove it. The results were good and I was quite satisfied with the entire cleaning!

Minor accident in the bottom left corner. Trace ripped off when removing pins... No problems otherwise! :)

Finally, I bought a can of PCB coating to prevent all those sanded traces and pads from oxidizing. It left a faint bluish tint on the treated parts of the board, but I guess it doesn't matter. It's not as if someone is going to look at this board every day.
I don't have a photo of it with just the spray coat applied, but you'll see what it looks like soon.

Rebuilding

Ordering parts

During my disassembly process I documented every part I removed and I had put an order from Mouser for most of the parts. I couldn't find any 1S1588 diodes (at least for any reasonable price IIRC), but after looking at the datasheet I was certain that a regular 1N4148 would be a perfect substitute.
For the more exotic and obsolete stuff I had to order from eBay. For example the transistors were all 2SC1815GR and a few 2SA1015. These were most likely used just because they were cheap and common at the time and I could probably have replaced them using almost any generic NPN and PNP transistors. However, I played it safe and ordered some from eBay since they're a bit more of an active component than the diode and they were still easily obtainable and really cheap. I already talked about the RAM.

I got a nice surprise when I received what I had ordered from Mouser. The 40-pin socket for the CPU looked like this:

Look! It's a 38-pin socket!

 Mouser got this picture mailed to them and then they sent me a new socket :)

Populating the board

Again, there isn't much to say about the actual rebuilding process and I actually didn't take that many photos of it, but here are a couple:

New sockets for CPU, 2 logic ICs, RAM and EPROM.

Note that in the picture above (and below) you can see a dark area.

That's where the protective spray coat was applied.

Trigger transistors area. Shiny new resistors!

Molex connector crimping in progress! I actually had to buy the tool for it as I had never done this before.

My first crimped connector! :)

Headers for the ribbon cables around the battery area.

Another header and some IC sockets.

Meanwhile in the living room...

Testing and repairing

When the board had everything put back I did the Lithium battery mod mentioned earlier, but I didn't mount the battery yet. The mod takes care of the problem with dangerous voltage to the RAM as well.
I powered it up... Silence... Tempo LED stayed lit whatever I did.
Noooooooo! I broke it!!!

I started probing with the oscilloscope, looking for problems with triggers, clocks and whatever I could think of. It seemed like everything was working. Why was it silent??? But... I hear something very silent... ?
After a while I realized I had pressed all the "cancel voice" buttons and the volume sliders for all the "add voice" sounds were zero. I had also plugged the cable into the high-Z output, making whatever sound that was left almost impossible to hear... Idiot!!!!

I uncancelled the sounds and increased the "add voice" volumes. It worked!!! There was sound!!
Wooohooo! I didn't break it!

RAM and battery backup

I tested the machine with the old RAM chips first, and tried to record a pattern. Didn't work. The recorded pattern was just garbage and sounded like random triggers.
I removed them and put the chips I got from eBay instead.
I used my bench power supply and a multimeter to check the expected battery current being drawn by the new RAM when the CR-78 was switched off. I found it to be 25nA, which is nice, but also an extremely unreliable value as it's almost the minimum value the meter could show. At least it showed that there was no short-circuit :)

I then tried to record a pattern. It worked! Yes!!!

I soldered the new Lithium battery to the board, recorded a pattern and then switched it off, waited a bit, switched it on again.
Pattern was gone. What?!

I made sure the mod I had done was correct. It was. Battery had 3.6V still. Were the chips broken anyway?
Hmm.. The memory didn't have any voltage on their VCC-pins...

It took me a while, but I found out the through-hole plating in the pads where the battery was soldered in was either damaged, or perhaps they were never through-plated to begin with. The traces that connected the battery to the RAM ran on the TOP side of the PCB and I of course had only applied solder to the bottom.
I applied some solder on the top side and tried again. Now the battery backup for the RAM worked fine! :)

Tempo LED

As I noticed when I thought the CR-78 was broken during my first test, the tempo LED wasn't blinking. Of course it turned out to be a wonderful coincidence. After some probing the the oscilloscope I could see that the flip-flop IC 115 (74C175 / CD40174, it had both numbers printed on it) that was responsible for controlling the LED did get the correct data, but didn't produce any output, so it was was broken. I replaced it and got the LED working again. Nice!

Recording problem

The last problems I found was when I tested the recording function properly. I went through all the instruments and recorded them onto separate tracks as varied as I could.

Now... Here's the problem with waiting too long with documenting what you have done...

I know that I had problems with recording all different instruments. I _think_ one problem was that some instruments just wouldn't get recorded. Again, I probed with the scope to find why it didn't work, all the way back to the RAM, via the CPU to the switch. After a while, I found I had an open circuit between an IC and a switch. This was definitely caused by the battery electrolyte corrosion.

See it? Second trace from the left, just a bit above where the 3rd trace begins.

Once that was fixed, I still had problems with recording...
Some sounds were recorded as Low Conga instead of whatever they were supposed to be recorded as.
Now... I'm not sure which symptom was, since again, it was a few months ago, but after another lengthy analysis, I found that the instrument selector switch had two open circuits... Urgh!

So, I had to remove that horrible 11-position switch...
Here it is.

The switch looked like it was built from customizeable parts (note the black blind sheet at the rear) and should be possible to disassemble. Turns out it was quite easy if you just bent those four tabs on the front so they fit through the holes behind them.

Here's the selector partially disassembled.

You can see some of the traces have blackened, but I don't think there was any problems with this one.
The plate behind it though... There was supposed to be open circuits on that one and judging by all the blackened traces and the miscolourations, it's probably also damaged by corrosion.


Looking at the middle switching plate after it's been removed, it's obvious...

I tried to find where the open circuit was. I expected it to be between the contact tabs and the rotor tab, but surprisingly, the problem was at the riveted joint between the trace and the contact tab.
Look at it from the side...

It appears that the potassium hydroxide gas had found its way in BETWEEN the trace and the contact tab, causing open circuit at the rivets.
I gave the switch plate a nice treatment with vinegar and alcohol (sounds like a bad party) and without doing anything with the actual problematic rivet joints, the open circuit was fixed!

Looks a LOT better now as well, don't you think?
But, since the connection might go bad again at the problematic joints I decided to solder it together, just as an extra precaution.

Short pieces of resistor legs, soldered in place, were used to bridge the connection once and for all.

I put the selector back together, soldered it back onto the PCB and tested recording, and everything was working perfectly!

Actually, the start/stop button was not 100% reliable, but I sort of gave up on that. I don't think it can be opened and I doubt I'd be able to find a replacement. In any case, I had had enough of this machine now and I was just happy that it actually worked and the electrolyte was cleaned off!

New power plug

The last thing I did was to replace the weird old (danish?) un-grounded wall plug that was attached to the mains cable. The cable DID have a ground wire that was attached to the case in the CR-78, but the plug didn't have any ground. Hardy a legal job I'd say, but I guess whoever didn't couldn't care less...
There's no photo of it unfortunately, but if you're desperate to know what it looks like you can see a part of it in the hi-res photo at the bottom, in the top left corner ;)

Done!

To round this massive post off, I'll add some more photos of the CR-78 after assembly.
Note that you can see IC 115 (the flip flop I had to replace to get the tempo LED working).
Next to the trigger transistors in the center of the board, there are 4 ICs.
One of them is socketed. That's the one...

And finally, a shot of the entire board from above, with the masking tape removed from the ribbon cables ;)
(I almost forgot to do that before I gave it back to the owner)

The entire repair took about 3 months.
(2013-02-11 - 2013-05-27), though a lot of the time I was just waiting for components to be delivered.

Also, I realize I've just spent 6 hours writing this blog entry... :-/