Using rsync to create a complete OS backup/clone sufficient?

It can't be done with rsync alone. You should first make a backup of the entire system. I personally use an rsync wrapper (Butterfly Backup) to do this. Here is an example:

$ bb backup --computer mypc --destination /backup --data System --type Unix

Then you should install a new machine and do a restore like this:

$ bb restore --catalog /backup --last --computer mypc

This way you brought the necessary files (passwd, etc) to the new system without touching special files like /dev

I think you are asking two different questions.

Will rsync copy/duplicate a file system? Yes it will. It can also be used to convert between file systems (you can copy an ext3 filesystem into a ext4 partition for instance). The file systems need to support each others features: copying ext4 into fat will lose a lot of functionality. You do need to mount the file systems; I am assuming you know how to do that. You also can usually get the live CDs to mount the file systems for you then press Control-Alt-T to open a command prompt.

I'm not familiar with LUKS; I expect that if you can mount the filesystem such that the files can be read that it will copy.

Your original question also asks about a clone. The copy you make with rsync will almost certainly not boot. You need to do a couple more steps.

If the partitions have changed, you will need to modify /etc/fstab. If your existing system is using UUIDs to identify the partitions, you will need to modify fstab. You can change it back to the old non-UUID style if you want; that makes it easier to clone but it is not recommended.

Then you will need to add boot code. You will need to use chroot to modify the target system. Most commonly grub2 is installed. The best examples I've found are here: Ubuntu community help on grub2

And because links move around, I'll gratuitously copy some examples:

this is a way to install grub

sudo mount /dev/sdXY /mnt # Example: sudo mount /dev/sda5 /mnt
sudo grub-install --boot-directory=/mnt/boot /dev/sdX # Example: sudo grub-install --boot-directory=/mnt/boot /dev/sda

And to update the directory of images in grub:

for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done
sudo chroot /mnt
sudo update-grub

Actually you can do it with rsync. But why bother doing it manually?

To make a complete copy of the OS the best tool would be Timeshift (it is offered by Debian and its offsprings, probably others too). It will create a complete copy of everything needed for a full backup-restore. Under the hood, the Timeshift will use rsync, but it already knows all pieces of OS which are required for complete copy.

No, it won't work at all.

You should send it to a mounted filesystem, so not /dev/[backup disk] but something like /mnt/[backup disk]/ after doing mount /dev/[backup disk] /mnt/[backup disk]

Furthermore, if you have multiple 'real' filesystems (ext4, btrfs, zfs, ...) you will also miss those. If you run mount | grep /dev/sd you can, in most cases see all the ones you have.
(There are exceptions: filesystems that are not mounted at the moment, filesystems that are not a /dev/sdX device, ...)

Edit: I didn't answer the 2nd question. The same problem here, and the same fix would apply.
BUT: If the destination filesystem doesn't use encryption everything will be saved decrypted