Rotation not synced correctly

Hey guys, I am currently having a problem with getting the correct avatar rotation to sync across players.

(note; I am using Unity 5.4 with SteamVR with HTC vive)

Players can join Rooms via Photon perfectly fine, when joined they instantiate an avatar prefab. This avatar prefab syncs the position just fine of the players’ head, and the rotation of your own avatar works aswell. But the rotation of the other player is doing unwanted stuff. Because the of the other player is the inverse of my rotation (e.g. when I look right he looks left and vice versa).

This is my script that I used for the avatar prefab:
using UnityEngine;
using System.Collections;

public class AvatarCopyScript : Photon.MonoBehaviour {

// Update is called once per frame
void Update () {
if (photonView.isMine)

transform.position = Camera.main.gameObject.transform.position;
transform.rotation = Camera.main.gameObject.transform.rotation;

}
}

At the avatar prefab I enabled at Photon Transform View: sync position, rotation and scale.

Thank you in advance
Pjtor