May I chime in?
What I see regarding hardware are two big questions that have to be answered:
1a) Keep the old case and with it the format of expansion boards, or make a true motherboard replacement
1b) Move to a new form factor such as ITX or similar
2a) Design using off the shelf CPU plus relevant peripherals and CPLD or similar 'glue' logic
2b) Go the FPGA route
Each has their own advantages and disadvantages.
Currently 1a + 2a is covered with the Tetroid SGC clone. The problem here is the Ingot CPLD which is becoming impossible to find and to a lesser extent the floppy controller chip.
Also, 1b + 2b os covered by the Q68 which is it's own single board computer and has no ties to old hardware except virtually
If we explore all the combinations, a bunch of possibilities are there, but in some cases there are serious hardware constraints.
For instance, given the old motherboard, the old ULA and IPC chipset is a serious constrait, even though surprisingly, the old 68008 CPU is not - 68SEC000 is still available at clock speeds of up to 20MHz (and seems eminently reclockable) and it can actually be set up to run on an 8-bit bus. However, even though most of the old chips could be replaced by newer FPGA/CPLD/uC based designs, there is one area where the 8-bit bus is a true ball and chain for any new development, and that is where speed counts - memory acces, and in particular, graphics.
While it is nice to dream about high resoultions and high color or better graphics, one must take into account that all of it is done by CPU and it comes down to how much data the CPU can move per unit time. Given that 512x256 in 4 colors is 32k, and the same resolution - now considered inadequate - at 256 colors is 128k, we would need at least a factor of 4 improvement in bus speed to just keep up with old QL style graphics, and even running the bus at the fastest 68SEC000 speed reaches about 5x and this is only because the old 8301 slows down the old 68008 to less than half speed. This is a major bottleneck, and one of the reasons Aurora was limited to (theoretically) 256k screen RAM - though running at roughly 2x 8301 speed (and capable of twice that but never used). That is not even enough for what is today considered the bare minimum, 640x480 in 256 colors. The only way to go further with an off the shelf CPU is to increase the bus width and with it also the speed of the CPU.
It should be said, however, that for most things except graphics and mass memory access, 8 bit wide buses (given somewhat of a speed boost) are perfectly adequate. It's even enough for a boot flash ROM. What this means is, assuming we are looking at an original QL style case, the system gets a definite pattern to it's hardware design - slower 8-bit peripherals to the right hand side, faster and wider RAM (and also video RAM) to the left hand side, which is also where some sort of expansion would be, if there is one at all. The CPU is in the middle.
The QL case is more or less structured that way except for the video output. If a more modern replacement motherboard is to be made, the proper place for that would be where the ROM port is (and the ROM port would become obsolete)
There are also constraints to using an off the shelf 68000 style CPU - there is actually only the one 68SEC000 available. While it is an exact implementation of a 68000 core, it is limited to the said 20MHz, 16-bit wide bus and a 16M address space. It only has 24 address lines, just like the original 68000. Given that this has to fit EVERYTHING, it is obvious that the maximum installed RAM can be up to 16M and in this case not all of it can be used as space has to be left for IO etc. Kind of tlike the SGC or GC, while they have 4 and 2M of RAM respectively, it cannot be fully used.
From this we have a fork in the road, leading to other, obsolete, but still available members of the 68k family.
One that is available but quite expensive is the 68328SZ Super-Dragonball 'system on a chip'. This one has a fully 68000 compatible core running at up to 66MHz and this is as fast as a 68k core from Motorola ever ran. The 68SEC000 uses a derivative of this core. The chip comes in a 256 pin uBGA package which is VERY impractical for amateur builds, but contains TONS of peripherals, including a LCD controller which can also drive a digital output and convert to analog, but the resolution is limited to 640x480 if any decent number of colors is to be used. It does have a simple form of bitblit, though.
The rest are the well known 68020/30/40/60 parts. Unfortunately availability is sketchy and prices can be completely unreasonable. One part, however, is still feasible - 68EC030. These, though obsolete, can still be had for reasonable money in speed grades up to 40MHz, while the much rarer fully blown one running at 50MHz is very expensive. In fact, given that the 68EC030@40M tends to run just fine at 50M and is actually cheaper than 68020 variants, AND it supports dynamic bus sizing which lets it easily use various bus widths concurrently in the system (extremely useful if you are doing a SGC replacement!), AND it can also be interfaced to burst-capable RAM sich as SDRAM, it would be THE chip to use for this purpose, without breaking the bank. Given proper support for external hardware, this option actually runs significantly faster than the current Q68.
040, 060 and also the first generation ColdFire (MCF5102) are all 000 compatible, but like the 020 and 030 before them, need cache support and patches to the OS and in some cases other software. They do not support dynamic bus sizing so it has to be implemented externally, if needed. If a replacement motherboard is made, and the original ULAs etch are chucked out, this may not be needed at all. Of course, something like the 68060 is still the speed king in this realm.
Peter and others have made the important point re integrating graphics in a would-be SGC replacement. This goes without saying, the 8301 ULA should be forgotten. Trying to build something around it is far more trouble than it's worth and for a 80s-of-previous-century result. It goes without saying that a full motherboard replacement would have integrated enhanced graphics.
Before I conclude this line of reasoning, I should mention that with enhanced graphics, not only memory speed has to increase, but also memory size. Multiple application's windows now take more space simply because of higher resolution and number of colors. What it boils down to is, put as much RAM as feasible on whatever you design. Keep in mind actual RAM may be limited to 512M by some common and not well behaved software that uses the top 3 address bits in registers to keep track of extra data about what it is addressing.
What this boils down to is the following:
In the 'use of the shelf CPU' camp, there are three levels one can think about, either as an expansion, or (preferably) as a motherboard replacement:
1) 68SEC000 running as fast as possible (probably slightly over 20MHz, say, 7.5MHz x 3 = 22.5MHz or 8MHz x 3 = 24MHz) fully kitted out with maximum RAM + VRAM for video. Given some decent CPLD/small FPGA support this system is not as complex as one would first think. Basic specs would be something like 8M of RAM, 512k of VRAM for max 1024x512 in 256 color displays, and other peripherals to match. These days a SATA port is not impossible. This system would beat the SGC in RAM and any legacy QL system in graphics but not speed. As speed goes, basically multiply GC speed by the CPU clock speed ratio, between 1.23 and 1.5.
2) 60EC030 well appointed with either burst DRAM+VRAM (32M DRAM + minimum 1M and more probably 2M VRAM for up to full HD in 256 colors) or any amount of SDRAM that is practical to put on there, again running as fast as possible, plus assorted peripherals. If this was supported by FPGA logic, and connected to SDRAM running at twice the CPU speed, graphics can be incorporated into the FPGA and the same RAM can be shared between CPU and graphics. EC030 has the advantage of caches, which, though small, increase performance significantly when implemented as a QL since the software itself is small and densely coded by today's standards. The two-fold increase in data path widt and caching puts this over both SGC and Q68 performance. This could actually be implemented as a sort-of cross between the Q68 and a SGC, basically Q68 based FPGA logic widened to 32 bits but without the 68k core given that there is an external CPU. Estimated performance around SGC x2..3, depending how well the memory interface is done. Note: EC030 has the full 32-bit address bus so no memory constraints as far as it's concerned.
3) 68EC060 with SDRAM, other stuff similar to the above. Given that old peripherals can be completely replaced, no need for dynamic bus sizing. Given that an EC060 can be had at 66MHz and will run faster, this is another big step over the 030 in speed, probably slightly faster than Q60 at the same speed given the RAM is faster. The same basic hardware, only simplified (due to multiplexed bus) could be used with a MCF5102. The problem is, the MCF5102 can be had but the price is exorbitant. You are more likely to get a refurbished full 060 for less, and the fastest speed grade for the 5102 was 40MHz, which would put you near Q40 speed at the same clock, which, while a nice step up over 030, is still about 3x slower than 060 at say 66-75MHz. One big advantage of the 040, 060 and 5102 is the ability to simply implement a dual processor board. It does require some tricks with decoding, interrupts and cache management but hardware wise it comes to connecting two CPUs in parallel. Now, this would be a whole different game IF it peaks the interest of remaining programmers.
The FPGA route, where a 68k or better core is implemented in a FPGA is far more flexible but also potentially needs far more effort.
Because you can program an FPGA in all sorts of ways and there are various densities (or 'capacities') available in pin-compatible cases, it is impossible to plot a discrete road map for such an approach. So let me just speculate on the lowest and highest ends.
On the low end one could implement a small board that would replace the 68008 CPU on the original motherboard. This could hold a 68k core with a small cache, running at a much higher than standard speed, possibly implementing some extras such as 2 more address lines (using pins previously carrying the signals FC2 and E).
From here, on-board dedicated RAM could be added or some peripherals. The next step would be re-doing it for the QL bus, and adding graphics. In a way this is the oposite of 2) above, Q68 redesigned to interface to the QL motherboard.
On the high end, there is a FPGA implementation of a new 68k+ sompatible core called the 68080, which includes a FPU and extension to 64-bits. It is not as fast as the fastest still available coldfires (which I have decidedly skipped for reasons I'll get to below), but still about 3-4 times faster than the fastest 68060, possibly more, depending on how much you are willing to pay for the FPGA to implement it.
Finally... Coldfire. Since NXP bought Freescale, the Coldfire lineup is getting decimated and ARM parts are being preferred. Unfortunately, even though the latest Coldfires were almost completely 68k compatible - as they basically put back in all that they foolishly pulled out in the first place, essentially guaranteeing certain doom for 68k - it still has one major difference and that is a single stack pointer which cannot be trapped and software emulation used to get full 68k compatibility. I'm almost sure they did this deliberately
It is a pity as I am sure lots of people would have liked a 32-bit implementation of 68k running at 600+ MHz. Apparently even far larger communities of 68k based computers like Amiga and Atari have not figured out a way to use these parts, if they had I am sure we would see super fast CF based accelerators, rather than FPGA designs. Unfortunately CF is on it's way to being completely obsolete and even far before that, it was almost impossible to get the parts in small quantity for any reasonable price. While it might be possible to modify the OS to run on CF and use lots of patching for other software, the obvious question is, who would do this?
We might as well go the route of writing a really good ARM based 68k emulator and use a raspberry PI... a QPI rather than QPC?