If I login as jack, and run vncserver giving a normal password, then I don't
get this issue. This is the working instance:

# pargs -l 865
/usr/X11/bin/Xvnc :1 -desktop 'opensolaris:1 ()' -auth /jack/.Xauthority
-geometry 1024x768 -depth 16 -rfbwait 30000 -rfbauth /jack/.vnc/passwd -rfbport
5901 -pn 

and the existing failing one:

# pargs -l 729
pargs: Warning, target locale differs from current locale
/usr/X11/bin/Xvnc :0 -depth 24 -nolisten tcp '-rfbauth=/etc/X11/.vncpasswd'
-audit 0 -br -auth /var/lib/gdm/:0.Xauth

Unfortunately I don't have a tool to "decrypt" vnc password files to hand, but
I'm pretty sure the saved one is what is in xenstore. It's generated like this:

        PASSWD=`/usr/bin/openssl rand -base64 8 2>/dev/null | cut -c0-8`
        printf "%s\n%s\n" "$PASSWD" "$PASSWD" | \
            /usr/bin/vncpasswd $PASSWDFILE >/dev/null

What's happening is this: gdm is now started as the user 'jack' (and hence Xvnc
is). But we don't have permission to read the VNC password file, which is owned
by root:

$ tail -f /var/log/gdm/\:0.log 
...
Wed Mar 11 14:10:40 2009
 SConnection: Client requests security type VncAuth(2)

Wed Mar 11 14:10:48 2009
 SSecurityFactoryStandard: opening password file '/etc/X11/.vncpasswd' failed
 SConnection: AuthFailureException: No password configured for VNC Auth
 Connections: closed: 129.150.120.199::65149 (No password configured for VNC
              Auth)


Fixing the vnc-config method to chown the password file would make install
possible. However, it's also possible to enable the VNC server post-install.
It's
far from clear what we should do in that case.

So why did it work in build 101?

gdm ran as root in 2008.11

Another problem with using jack is due to bug 4215:

AI doesn't set 'root' role and 'Primary Administrator' profile


If jack does not have "root"  role, how will VNC works.

Should this be a duplicate of bug 4215 ???

No, root isn't required to run a VNC server (I've verified that chown()ing the
password file and restarting gdm is a work-around).

(In reply to comment #4)
> gdm ran as root in 2008.11


OK, so then the creator of .vncpasswd needs to be adjusted; that's done by
virt-install, isn't it?  If so, please transfer to appropriate xvm component.

Please provide input to Comment #7 From  Dave Miner

No, it's done by the xvm-vnc-config SMF method, which currently lives in
pkg-gate.

Also, chowning the file doesn't help the post-install case - what does gdm run
as then? Need input from alanc?

After speaking with with John Levon I'm reassigning this to the packaging
component.

Not me - try Brian Cameron, he's the GDM maintainer (GDM is GNOME, not X).

The "gdm" SMF service normally starts up GDM, so I am curious how the "jack"
user 
could be running GDM (as comment #2 suggests).  Although I am the GDM
maintainer, 
nobody ever informed me that there was any plans to change the owner of the GDM 
processes like this in OpenSolaris.

I just started up my OpenSolaris build 109 from CD and since GDM is configured 
with AutomaticLogin turned on (in /etc/X11/gdm/custom.conf), GDM does not show
the login screen, but instead just starts up the user session as 
the "jack" user immediately.

So, after the jack user logs in, I run "ps -ef | grep gdm" to see if the GDM
processes are running as jack.  I see that the /usr/sbin/gdm-binary processes 
(the GDM daemon processes) are correctly running as the "root" user.  So, I am
not sure why people are suggesting that GDM is running as the "jack" user.  It
does not seem to be.

I would not expect GDM to work at all if the GDM daemon processes were running
as any other user than root (unless perhaps that other user had the same
authority and permissions as the root user).  The GDM GUI programs should run
as the "gdm" 
user.

If I am misunderstanding you, please clarify what the problem is.

Sorry, I mis-spoke. It's the X server not gdm that's running as jack:

jack@opensolaris:~$ ps -ef | grep Xvnc
    jack   728   723   1 08:49:53 ?           0:00 /usr/X11/bin/Xvnc :0 -depth
24 -nolisten tcp -rfbauth=/etc/X11/.vncpasswd -audi

Ah, I suspect I know what changed.  Note that on Solaris, the Xserver supports
an interface to drop the Xserver down to user permissions for security reasons.
 So
GDM calls the interface after starting the Xserver to make Xorg drop to user
ownership.

Running the Xserver as root makes the system more vulnerable since anyone who
could exploit the Xserver could get root permissions.  This is obvious
undesirable, and the reason why Solaris supports this interface.  Solaris has 
worked this way for a long time.  GDM supports this in Solaris 10, and CDE
login in even older versions of Solaris.

In earlier versions of OpenSolaris, however, this was broken.  This feature
didn't work due to bug #5973.  This issue was fixed by making sure that GDM
creates the /var/dt directory if it is missing (which it was on OpenSolaris).
This was fixed in build 107.

So, the change in behavior you are seeing is probably due to this problem.

I suspect the right fix for this issue is to modify the ownership/permissions
of the vnc password file so it can be accessed when Xorg is running as the
user.

Thanks Brian. Unfortunately it's not quite that simple: whilst we can
change the script that generates the VNC login file to use 'jack' for the live
CD, there's still the post-install case.

How does this work in this circumstance? Presumably X must be running before we
know what user to become. Does it start up as root until somebody logs in via
gdm?

The password file generation happens way before gdm (or X) starts. Given the
above, would a fix that does something like:

# leave as root unless we're the live CD
if test -f /.livecd; then
    chown jack /etc/X11/.vncpasswd
fi

be the right thing?

(In reply to comment #15)
> Thanks Brian. Unfortunately it's not quite that simple: whilst we can
> change the script that generates the VNC login file to use 'jack' for the live
> CD, there's still the post-install case.
> 
> How does this work in this circumstance? Presumably X must be running before we
> know what user to become. Does it start up as root until somebody logs in via
> gdm?

Yes - the X server listens on a named pipe for a message from gdm/dtlogin
to tell it who to setuid to once the PAM conversation/authentication is
successfully completed.

> How does this work in this circumstance? Presumably X must be running before we
> know what user to become. Does it start up as root until somebody logs in via
> gdm?
>
> The password file generation happens way before gdm (or X) starts. Given the
> above, would a fix that does something like:
>
> # leave as root unless we're the live CD
> if test -f /.livecd; then
>     chown jack /etc/X11/.vncpasswd
> fi
>
> be the right thing?

I don't see how this could work properly.  Hacking it like this might make it
work for the "jack" user, but it wouldn't work for any other user.  You can't
know what user will be logging in until after GDM authentication.

I am not sure how VNC needs to work with its password files, but some ideas
about how you could fix this:

1) Note that GDM does have a PostLogin and PreSession script.  Both runs as
root after authentication but before running the user session.  This might be a
good hook to change the ownership/permissions of the file before starting the
session if that's early enough.  Though you'ld need to be careful to ensure
that when multiple users log in that the right user gets access to the file. 
For example, some users might log in via XDMCP or on a device in a Sun Ray
environment and you probably do not want to change the ownership/permissions of
the file each time any user logs in like this.

2) Another hook you could use is a PAM module that you could add to the PAM
stack if you wanted to change ownership/permissions on the file after
successful authentication.

Instead I would guess that you only want the user logging into the console or
DISPLAY=:0 (or something) to have access to the file.  So you probably need
some code to make sure that the file permissions are only changed when a user
logs into the display that make sense.

2) If the above approach isn't sufficient, then you probably need to make VNC
work with RBAC so that user can be configured to get authorization to access
the /etc/X11/.vncpasswd file.  The jack user could be set to have such
authorization by default, and other users could be dconfigured to get
authority.  Or you could use the "ConsoleOwner" RBAC role to allow any user on
the console to have permissions.

Just some ideas of how you could approach this.

> I don't see how this could work properly.  Hacking it like this might make it
> work for the "jack" user, but it wouldn't work for any other user.

Hmm, yes, presumably even Xvnc will have become the user by this time. Ugh.

Looking at our options, it seems the best solution for now is to simply refuse
to work post-install.

(In reply to comment #17)
> I don't see how this could work properly.  Hacking it like this might make it
> work for the "jack" user, but it wouldn't work for any other user.  You can't
> know what user will be logging in until after GDM authentication.

But you can't get to gdm authentication until Xvnc has started and read the
password file *UNLESS* you're using auto-login, like the LiveCD does - so isn't
this only an issue for autologin?

I seem to remember from my testing that Xvnc reads the password not at startup
but when a connection attempt is made.

Perhaps this is only an issue for automatic or timed login.  Not sure. 
However, what if the user changes the automatic/timed login user to a different
user?  If they do this, things should "just work" if possible, I'd think.

John, this bug was discussed in the OpenSolaris engineering
Bug Court meeting today. David asked me to assign it to you,
as you are the best person to know what needs to be done here.

Thanks.

We should do the following two changes:

- chown the password file to jack:jack on the live CD
- remove the code that attempts to run in any other context

The latter is a little unfortunate, but given the uncertainty over
how X does or doesn't work, and the time pressure, I don't think we have
much option.

remote: added 1 changesets with 2 changes to 2 files
remote: Sending notification for 86ed8d62da8b329521beac72c9eba78dfb1b1a36