14:00:04 <bstinson> #startmeeting CBS/Infra 14:00:04 <centbot> Meeting started Mon Nov 10 14:00:04 2014 UTC. The chair is bstinson. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:04 <centbot> Useful Commands: #action #agreed #help #info #idea #link #topic. 14:00:23 <bstinson> #chair MerlinTHP alphacc Arrfab kbsingh 14:00:23 <centbot> Current chairs: Arrfab MerlinTHP alphacc bstinson kbsingh 14:00:29 <bstinson> #topic Agenda 14:00:35 <bstinson> #info Topic: Status Updates 14:00:40 <bstinson> #info Topic: Old Business: Ad-Hoc Upstream Development Repositories 14:00:57 <bstinson> #info Topic: Open Floor 14:01:14 <bstinson> hi everyone 14:01:19 <Arrfab> hey bstinson 14:01:21 <MerlinTHP> ello 14:01:28 <alphacc> hi all 14:01:39 <bstinson> #topic Status Updates 14:02:04 <bstinson> Arrfab: you had something regarding the requierments list? 14:02:42 <Arrfab> bstinson: for Central Auth ? yes : MerlinTHP was supposed to write it but I just collected some ideas and start a wiki page so that everybody can subscribe/track/edit 14:02:59 <MerlinTHP> OK 14:03:13 <MerlinTHP> I'll work out if I can edit the wiki, then merge in the stuff I've got 14:03:16 <Arrfab> #info wiki page around Centralized Authentication : http://wiki.centos.org/InfraWiki/CentralizedAuth 14:03:21 <bstinson> great! 14:04:04 <mikem> good morning 14:04:08 <Arrfab> but the more I think about it, the more I think that IPA is the way to go, as long as we don't use the integrated dns feature, nor expose the KDC to the outside world 14:04:35 <Arrfab> MerlinTHP: I can add you to that page in the acl, as soon as I know your wiki login 14:04:37 <MerlinTHP> OK, so on the IPA side, I've been told that we'll need something sooner rather later, so I've been working on the not-a-PoC version of the IPA front end. 14:04:41 <MerlinTHP> Arrfab: ok 14:05:11 <MerlinTHP> It's not ready yet, but people can start laughing at my python code at https://github.com/merlinthp/MyPA 14:05:19 <Arrfab> MerlinTHP: cool, you have access to a Poc machine, right ? so is there something we can see/test "live" ? 14:05:25 <alphacc> MerlinTHP: great 14:05:27 <MerlinTHP> Yep. 14:06:03 <kbsingh> woo 14:06:09 <MerlinTHP> One of the things I'll try and prioritise is fleshing out the README with architecture similar. 14:06:11 * kbsingh reads backlog 14:06:42 <MerlinTHP> #info Work in progress IPA front end code at https://github.com/merlinthp/MyPA 14:07:03 <bstinson> nice! let's bring MyPA up as an example when we talk about Ad-Hoc Upstreams later 14:07:09 <MerlinTHP> Sure 14:07:15 <Arrfab> MerlinTHP: good README.md about the fact that the default HBAC "allow all" exist : I've been surprized the first time I tested ipa-server myself :-) 14:07:21 <MerlinTHP> Arrfab: :) 14:07:32 <MerlinTHP> Yeah, that's a good thing to IMMEDIATELY DISABLE 14:07:36 <Arrfab> yeah 14:07:58 <MerlinTHP> I can see why it exists. Without it IPA would look like it didn't work when you first tried it. 14:08:30 <MerlinTHP> And the number of times I've forgotten to add a new system to a hosts group, so I can't log in, then wonder why... 14:08:31 <Arrfab> so on a side note, we discussed about the fact that ipa-server would be the one included in el7/c7 meaning that , if we start with c7, those nodes will be managed "manually' as the current puppet version in centos.org doesn't support it (yet) 14:08:50 <MerlinTHP> Yeah, I'm doing all my dev work on c7 14:09:30 <MerlinTHP> Anyway, hopefully I'll get some more work on it done over the next few days, depending how much time the new Halo takes up 14:09:53 <bstinson> :) 14:10:10 <MerlinTHP> That's me, really. 14:10:56 <bstinson> anything else for status updates? 14:11:08 <kbsingh> I've done some work on the gitblit upgrade as well 14:11:25 <MerlinTHP> Is this the one that'll fix the extra / issue? 14:11:55 <kbsingh> yeah, that should get fixed - but more importantly, this also has the username/branchname mappings 14:12:07 <MerlinTHP> Ah :) 14:12:09 <kbsingh> I need to do a bit more work to make sure the failover machine is all setup to take up load and then we can flip over 14:13:02 <kbsingh> at some point in the near future we should get dev.git.centos.org up again and use that to work on the auth integration stuff 14:13:25 <MerlinTHP> *nod* 14:13:51 <alphacc> #info I have been testing the iamge creation from koji with imcleod. It seems it works fine with 1.9.0 14:14:00 <kbsingh> the other big win from the gitblit upgrade is that the new api has a bunch more features in there 14:14:18 <kbsingh> once that is done, we can then bring back the ban on people randomly scraping the site from top to bottom 14:14:42 <bstinson> that would be handy 14:14:43 <kbsingh> at the moment, were doing millions of hits a day, most of which just seem bizaree 14:14:49 <bstinson> alphacc: woohoo! 14:15:35 <alphacc> there are some more tests needed for atomic images, hope we can continue this week. 14:15:55 <Arrfab> alphacc: good news 14:16:07 <alphacc> iamge factory very well integrated with koji, hope the docker plugin will work too. 14:16:11 <bstinson> #info centpkg hasn't moved forward much this week, but kbsingh gave me access to the repo (which I still need to test) 14:16:46 <alphacc> imcleod is the plugin owner so I expected to see details with him. 14:16:57 * imcleod perks up 14:17:11 <imcleod> alphacc: Looking for docs/details of the image building in koji? 14:17:23 <imcleod> alphacc: Was going to send something about that to centos-devel today. 14:18:11 <alphacc> imcleod: excellent, please do. 14:18:26 <imcleod> alphacc: Is the code on the CBS newer than what we worked with on Friday afternoon? 14:18:56 <alphacc> imcleod: no not yet. 14:19:06 <alphacc> imcleod: let's talk about it after the meeting 14:19:30 <kbsingh> so ifyou guys are changing code on there - be good to see a note on centos-devel with highlights of what changes 14:19:44 <kbsingh> we should try and do that all around really, just as a good practise 14:20:00 <bstinson> +1 14:20:05 <alphacc> yes 14:20:23 <imcleod> kbsingh: Roger. The work on Friday was not a code change, just feature-enablement (which is not actually a word, according to my spell checker). 14:21:19 <kbsingh> heh 14:21:39 <kbsingh> a slightly related question - what machine are you going to use for the image builders ? 14:21:56 <kbsingh> is that just a regular koji job so the regular builder will do its thing, or do / did we need something special there 14:22:04 <alphacc> kbsingh: so far we have 1 builder, bl7. 14:22:07 <MerlinTHP> It's the former 14:22:16 <MerlinTHP> It's just another thing kojid does 14:22:28 <imcleod> Correct, though it does require some additional packages to be installed. 14:22:33 * MerlinTHP nods. 14:22:39 <alphacc> and it has a channel associated so we can choose builder that do only this task 14:22:45 <imcleod> And it works _much_ better if the builder in question has HW accelerated virt enabled. 14:22:52 <MerlinTHP> I'd quite like to get some roundtuits to properly integrate mashing as a koji task too 14:23:08 <imcleod> MerlinTHP++ 14:23:16 <alphacc> MerlinTHP: +1 I am looking at that too 14:23:24 <MerlinTHP> mash as it is is a horribly inefficient process. 14:23:49 <MerlinTHP> alphacc: ok, remind me sometime and I'll pull together somes notes on the hacks I made to $work's mash 14:24:03 <imcleod> dgilmore has had "optimizing mash" as a long term TODO for some time now, I believe. 14:24:03 <MerlinTHP> It's not in-koji, but it's a lot faster than normal mash 14:24:38 <bstinson> speaking of roundtuits, MerlinTHP: can you link me the upload.cgi stuff you posted a few weeks ago? 14:24:46 <MerlinTHP> Yeah, sure 14:24:46 <alphacc> MerlinTHP: Ok I'll share our stuff too 14:24:53 <imcleod> MerlinTHP: I'd be quite interested in those as well. I was browsing the code on the flight back from Paris and had some thoughts. 14:24:56 <kbsingh> so: something to track: we have potentially more machines we can bring into builder roles 14:25:26 <kbsingh> ~ if / when we hit a point where people need to wait > 30 min for their package to build start, then we should visit that and see if we can bring in more h/w 14:25:41 <MerlinTHP> #action MerlinTHP to write/share some notes on more efficient mash 14:26:08 <alphacc> #action Identify more builders and install them with puppet. 14:26:11 <kbsingh> secondly, lets setup a section on wiki.c.o that everyone here can use to track / document / edit / howto / tips nad tricks etc around the buildsystem 14:26:19 <MerlinTHP> kbsingh: otoh, that builder is a fairly beasty blade, it can run a bunch of builds in parallel 14:26:35 <MerlinTHP> bstinson: here ok? http://www.merlinthp.org/centos/lookaside/ 14:26:36 <kbsingh> MerlinTHP: yeah, so lets only look at more hardware when we need to, not needed right now 14:26:40 <mattymo> Evolution, are you around? 14:26:45 <MerlinTHP> bstinson: or do you want a mail or something? 14:27:05 <alphacc> kbsingh: I'd like to go forward with a puppet install if we can have additional blade. What do you think Arrfab ? 14:27:06 <bstinson> MerlinTHP: that's perfect, i'm going to put that in a repo somewhere 14:27:42 <alphacc> so we are ready when we need more hw. 14:27:45 <kbsingh> ther might be some dancing involved in finding more blades, but it can be done . i dont want to derail this meeting though 14:27:53 <Arrfab> alphacc: all for it 14:28:41 <MerlinTHP> Arrfab: do you want a ticket re: wiki perms? 14:29:01 <Arrfab> MerlinTHP: no, don't think so .. just give me your login name :-) 14:29:10 <Arrfab> no need to track ACL right for wiki :-) 14:29:12 <MerlinTHP> HowardJohnson 14:29:14 <MerlinTHP> ta 14:29:40 <bstinson> ok, anything else for status updates? 14:29:53 <Arrfab> #action arrfab to give permission to HowardJohnson to the CentralizedAuth wiki page 14:30:10 <MerlinTHP> ta 14:30:12 <Arrfab> bstinson: don't think so 14:30:33 <bstinson> #topic Old Business: Ad-Hoc Upstream Development Repositories 14:30:39 <kbsingh> if we are still doing updates, i have a question re: lookaside uploader. 14:30:45 <MerlinTHP> Shoot 14:30:47 <kbsingh> are ew blocked on having a cert / auth layer in place 14:30:52 <kbsingh> or are we able to push ahead with that 14:31:02 <MerlinTHP> Not really blocked, we can use the existing koji CA 14:31:12 <kbsingh> its a primary blocked ( not having the lookaside uploaders ) to getting mass git usres onboard 14:31:24 <MerlinTHP> I was going to ask you if what I'd implemented was ok for user-branch mapping 14:31:36 <kbsingh> i didnt know what you implemented and where 14:31:39 <MerlinTHP> And, obviously, failed 14:31:45 <MerlinTHP> Yeah, that too 14:32:04 <MerlinTHP> If you've got a few minutes after this meeting? 14:32:57 <bstinson> MerlinTHP, kbsingh: if you're going to work on that, i might leave the repo creation to you 14:33:04 <bstinson> but we should get that in git somewhere 14:33:28 <MerlinTHP> There's a repo on git.c.o it should go in 14:33:38 <kbsingh> today is call mayhem monday.. but we can try and work something out 14:33:44 <MerlinTHP> cbs-tools 14:33:57 <MerlinTHP> kbsingh: ok, we'll have a go 14:34:00 <alphacc> +1 14:34:16 <kbsingh> maybe we can use email to work this out - on the centos-devel list 14:34:21 <MerlinTHP> TBH, it's 2:30 in the afternoon and I'm still eating my lunch 14:34:24 <MerlinTHP> kbsingh: k 14:34:30 <kbsingh> i just had my lunch at 1:55 :) 14:34:56 <imcleod> Thanks to jet lag, I feel like eating lunch but it is only 9 AM. 14:35:19 <Arrfab> ah, it's that moment in the meeeting when we start discussing about lunch .. I guess next step is beer :-p 14:35:20 <bstinson> i'm about 1/4 of the way through my first coffee (the day is looking much brighter) 14:35:38 <MerlinTHP> Arrfab: Timmermans, please. 14:36:04 <Arrfab> bstinson, MerlinTHP, kbsingh, imcleod (and all others too) : feel free to read/edit/comment the requirement list on the wiki page around IPA/FAS please 14:36:23 <Arrfab> #link http://wiki.centos.org/InfraWiki/CentralizedAuth 14:36:32 <bstinson> Arrfab: cool 14:36:32 <MerlinTHP> Will do 14:36:35 <imcleod> Arrfab: Many thanks. Will do. 14:36:52 <bstinson> Let's talk about Ad-Hoc upstreams 14:37:09 <Arrfab> I had some calls with some of the people involved in FAS, so we can "use" that knowledge, even if we want to use IPA 14:37:31 <MerlinTHP> Cool. 14:37:49 <Arrfab> like the FedOAuth system, written by Fedora guys, but being able to use multiple backends providers, so FAS, but also kerberos/ldap/other -> IPA ready too :-) 14:38:15 <kbsingh> bstinson: in principle i think we all agreed on having them 14:38:29 <kbsingh> bstinson: i guess the bits we need to workout are how to use them and hoto consume various parts of these upstreams 14:38:43 <Arrfab> meaning than most things supporting OAuth (probably phpbb for the forums and I have to verify for Mantis/bugs.c.o) would be able to integrate with IPA 14:38:56 <kbsingh> my understanding initially was that people would keep their dist-git repos in rpms/ project and use the upstreams stuff in their own sig-specific project area for adhoc content 14:39:26 <kbsingh> and what that does seem to be fine, people also want adhoc dist-git git repos in the upstreams areas - and to me thats confusing as to where koji is going to get its git tree's from 14:39:59 <kbsingh> eg. if someone wants to build an rpm from sig-blahblah/kernel.git; how is that going to map to what we already have in terms of expectations ( ir. branch names / sig names etc ) 14:40:05 <MerlinTHP> If people want ad-hoc branches in g.c.o dist-git, that's one thing. 14:40:08 <bstinson> hmmmm, i'm not sure if we want to consume arbitrary dist-git repos 14:40:15 <MerlinTHP> We don't. 14:40:22 <MerlinTHP> Not least because it requires koji config. 14:40:25 <bstinson> i think if it builds in the buildsystem, it should be under rpms/ in our branch layout 14:40:26 <MerlinTHP> Plus... just no. 14:40:50 <kbsingh> arbitary.. as in its still on git.centos.org, just in the sig's own project rather than in rpms/ 14:41:23 <Evolution> I think /rpms should stay separate for rh->push into the project. 14:41:37 <Evolution> but there should be something similar for project specific upstream/packages 14:42:00 <bstinson> so we need something like /sig-rpms? 14:42:19 <Evolution> well, it looks very much like we're going to be the upstream for software collections 14:42:34 * MerlinTHP puts a paper bag over their head 14:42:37 <Evolution> if we're doing one, we should plan for others. 14:44:10 <kbsingh> the other problem with dist-git is that we have no way to upload non text sources. 14:44:34 <kbsingh> and the user acl will map all rpm repos to force signame -> branch name 14:45:26 <kbsingh> so in a nutshell, even if we were to allow this - and realistically, if we have generic git repos,users can store whatever - even dist-git style stuff without needing any special foo 14:45:42 <kbsingh> just that they will need to use the srpm route to build in cbs.centos.org - there wont be any direct link from koji 14:45:58 <Evolution> how terrible would two separate git instances be? git.centos.org for official sources. rh-> us 14:46:17 <Evolution> then dev.centos.org or projects.centos.org or whatever for sig/upstream/etc git? 14:46:54 <kbsingh> for ths scl upstreams they dont need a direct koji link up do they ? 14:47:24 <kbsingh> afaik, they dont even want a git repo per rpm, they want a git repo per collection that totally destroys our ability to build from it 14:47:33 <kbsingh> since each git repo will have upto 200 spec files 14:47:43 <kbsingh> not sure if koji cam handle that 14:48:15 <imcleod> kbsingh: Do you already have patches in koji, or pending, to enable the structure you are proposing? 14:48:31 <imcleod> alphacc: I know you showed me a few things in the commit history on Friday. Are those related to this? 14:48:36 <kbsingh> no 14:48:42 <kbsingh> also what do you mean by 'this' 14:49:15 <kbsingh> cbs.c.o can consume git repos from the rpms/ project. so i am guessing its already there, mikem was working on that ages ago with alphacc 14:49:19 <imcleod> kbsingh: "this" = The proposed structure for dist-git repos on b.c.o. 14:49:29 <kbsingh> i believe the underlaying rpkg code has support now for the centos lookaside sources as well 14:49:41 <imcleod> kbsingh: b.c.o = git.c.o 14:49:41 <kbsingh> imcleod: not proposed, its in production 14:50:00 <imcleod> kbsingh: Ahh. OK. 14:50:06 <kbsingh> its where we get our upstream sources as well, so whatever there is now is what we've got for the short haul 14:50:33 <kbsingh> so to recap, and jus make sure we are all on the same page 14:51:10 <kbsingh> the rpms/ project on git.centos.org runs our dist-git stack, and branch names are considered 'sacred?' where branchname = c4/5/6/7/8 maps to the distro content, and sig's can branch to their own name == branch name 14:51:42 <kbsingh> then additionally, we have the signame projects, where they can have git repos for anything, where they get to do whatever branches / tags they want == these are the upstreams git repos 14:51:58 <alphacc> kbsingh, imcleod : we just want to roll our own koji package with the patches. nothing else 14:52:05 <kbsingh> however, we might not be able to link cbs.c.o to these Upstreams - and therefore if anyone wants to build rpms from these repos they need to use the srpm route 14:54:06 <bstinson> been thinking on this for a while, i'm not sure there's any other way 14:54:27 <kbsingh> yeah 14:55:14 <bstinson> of course i'm not particularly familiar with the SCL build process 14:56:51 <MerlinTHP> OK, we're running short of time. 14:57:01 <kbsingh> lets punt this over to the centos-devel list, and use it as a point to get started from 14:57:04 * MerlinTHP coughs and gives bstinson his hat back 14:57:14 <kbsingh> i dont think there is a clear solution to this in the short term that might result in anything different 14:57:17 <bstinson> heh 14:58:02 <bstinson> kbsingh: maybe we can hash out the rest of the use-cases also (and pick some of the low-hanging fruit) 14:58:08 <kbsingh> cool 14:58:21 <kbsingh> Ill get on the SCL list and rattle a few bolts around 14:58:30 <bstinson> our CBS-related tools for example 14:58:51 <bstinson> ok now for a very short... 14:58:54 <bstinson> #topic Open Floor 14:59:23 <MerlinTHP> 60 seconds, go ;) 15:00:08 <bstinson> thanks everyone! 15:00:21 <bstinson> #endmeeting