Bugzilla – Bug 6877
pkg search -r only shows packages up to build 106 for pkg.opensolaris.org
Last modified: 2009-03-03 11:19:51 UTC
You need to log in before you can comment on or make changes to this bug.
This is found when I do AI testing. I wanted to install a certain version of osol via AI. This had also been seen by Sean McGrath and Jan Damorsky as well. I was able to do this with osol 106 by doing the following: <ai_manifest name="default"> <ai_pkg_repo_default_authority> <main url="http://ipkg.sfbay/dev" authname="opensolaris.org"/> <mirror url=""/> </ai_pkg_repo_default_authority> <ai_packages> <package_name> entire@0.5.11-0.106 SUNWcsd SUNWcs slim_install@0.1-0.106 SUNWinstalladm-tools </package_name> </ai_packages> </ai_manifest> I found those versions either via pkg search -r against http://ipkg.sfbay/dev or via the web interface in firefox by searching the approriate packages by accessing http://ipkg.sfbay/dev When I try to do this for 107 or 108, for some reason pkg search -r does not show whole content of repo for some reason. root@opensolaris:~# pkg search -r entire INDEX ACTION VALUE PACKAGE description set entire pkg:/entire@0.5.11-0.94 description set entire pkg:/entire@0.5.11-0.91 description set entire pkg:/entire@0.5.11-0.95 description set entire pkg:/entire@0.5.11-0.98 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.91 description set entire pkg:/entire@0.5.11-0.90 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.94 description set entire pkg:/entire@0.5.11-0.90 description set entire pkg:/entire@0.5.11-0.94 description set entire pkg:/entire@0.5.11-0.96 description set entire pkg:/entire@0.5.11-0.97 description set entire pkg:/entire@0.5.11-0.98 description set entire pkg:/entire@0.5.11-0.93 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.93 description set entire pkg:/entire@0.5.11-0.89 description set entire pkg:/entire@0.5.11-0.96 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.100 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.91 description set entire pkg:/entire@0.5.11-0.93 description set entire pkg:/entire@0.5.11-0.94 description set entire pkg:/entire@0.5.11-0.105 description set entire pkg:/entire@0.5.11-0.106 root@opensolaris:~# pkg authority AUTHORITY URL opensolaris.org (preferred) http://ipkg.sfbay/dev/ root@opensolaris:~# However, if you do the 107 install with the following custom AI manifest, it does install 107: <ai_manifest name="default"> <ai_pkg_repo_default_authority> <main url="http://ipkg.sfbay/dev" authname="opensolaris.org"/> <mirror url=""/> </ai_pkg_repo_default_authority> <ai_packages> <package_name> entire@0.5.11-0.107 SUNWcsd SUNWcs slim_install@0.1-0.107 SUNWinstalladm-tools </package_name> </ai_packages> </ai_manifest> This is bad because for a novice user, it will be good to confirm that the particular version of the packages/cluster they want to install is available before I started customizing my xml. In fact, there had been many people that asked me how to do the above and it will be bad if pkg search -r or web search of http://ipkg.sfbay/dev does not work correctly.
This is because the search indexes need updating: A user recently noted that search results only returned packages up to build 106 for the /dev repository. Upon further investigation, I discovered they were correct, and that is because the repository contains the full index up to 106 *plus* a partial index that was never completed for later builds. So, right now, ipkg.sfbay/dev's search results only have data for up to build 106 and pkg.opensolaris.org/dev is the same. That means RE needs to remove the /tank/osol-dev/active/index/TMP directory, run the depot with --refresh-index and provide us with an updated incremental or something. It took about 10 minutes on ipkg.sfbay to run --refresh-index on a clone of /tank/osol-dev/active as an experiment, and it appears to have the desired effect. I suspect the possible RFE/bug here is that a depot server started in --readonly mode should fail and exit if it finds a partial index (e.g. /tank/osol-dev/active/index/TMP) to ensure this gets noticed again in the future. As for how this happened? I can only assume someone started a depot and cancelled it halfway through its normal indexing process. Maybe Brock has some other ideas. As for pkg.opensolaris.org, I'm not sure how Dan wants to handle that.
I had included the RE people in this bug to see how they want to handle this in ipkg.sfbay. This might impact other ipkg mirrors as well. If RE needs to do this as a regular process when they update ipkg.sfbay, they need to know and had a formal process to make sure that this is done.
(In reply to comment #2) > If RE needs to do this as a regular process when they update ipkg.sfbay, they > need to know and had a formal process to make sure that this is done. The only formal process here is to be certain the depot server being published to (if using a http:// repo) is done building search indices. A message should be printed to the depot log when search indexes are complete. In this case, it's very likely that the pkg.depotd server was still building search indices after publishing completed and so when the snapshot was created of the repository directory, a partial index was part of the snapshot. Long-term, we need to change the importer to block completion of the process until search indices are finished building. However, that will depend on the implementation of bug 6572 and bug 6573. As a workaround, if you publish to the repository using file://repo_dir instead of starting pkg.depotd, it will block until the search indices are complete.
It looks like someone had fix this on ipkg.sfbay/dev after I filed the bug: jack@line1-x4100:/export/home/profiles/osol_ai# pkg authority AUTHORITY URL opensolaris.org (preferred) http://ipkg.sfbay/dev/ jack@line1-x4100:/export/home/profiles/osol_ai# pkg search -r entire INDEX ACTION VALUE PACKAGE description set entire pkg:/entire@0.5.11-0.94 description set entire pkg:/entire@0.5.11-0.91 description set entire pkg:/entire@0.5.11-0.95 description set entire pkg:/entire@0.5.11-0.98 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.91 description set entire pkg:/entire@0.5.11-0.90 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.94 description set entire pkg:/entire@0.5.11-0.90 description set entire pkg:/entire@0.5.11-0.94 description set entire pkg:/entire@0.5.11-0.96 description set entire pkg:/entire@0.5.11-0.97 description set entire pkg:/entire@0.5.11-0.98 description set entire pkg:/entire@0.5.11-0.93 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.93 description set entire pkg:/entire@0.5.11-0.89 description set entire pkg:/entire@0.5.11-0.96 description set entire pkg:/entire@0.5.11-0.86 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.99 description set entire pkg:/entire@0.5.11-0.100 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.101 description set entire pkg:/entire@0.5.11-0.91 description set entire pkg:/entire@0.5.11-0.93 description set entire pkg:/entire@0.5.11-0.94 description set entire pkg:/entire@0.5.11-0.105 description set entire pkg:/entire@0.5.11-0.106 description set entire pkg:/entire@0.5.11-0.108 description set entire pkg:/entire@0.5.11-0.107 jack@line1-x4100:/export/home/profiles/osol_ai#
Yes, Dan worked around it. See his email to re.
I have modified the RE build process to ensure that the TMP directory is removed and that the repository index is refreshed before creating a ZFS snapshot and putting the repository into read-only mode. Is this the only required fix for this bug? If so, I can mark it as resolved.
Yes, that is the only change needed. Feel free to close this with yourself as the assignee.
*directed at Alan Steinberg*
Fixed in our process documentation.