15:06:58 RRSAgent has joined #social 15:06:58 logging to http://www.w3.org/2017/07/12-social-irc 15:07:00 RRSAgent, make logs public 15:07:00 Zakim has joined #social 15:07:02 Zakim, this will be SOCL 15:07:02 ok, trackbot 15:07:03 Meeting: Social Web Working Group Teleconference 15:07:03 Date: 12 July 2017 15:07:05 present+ 15:07:07 present+ 15:07:07 present+ 15:07:09 present+ 15:07:11 scribenick: puckipedia 15:07:13 present+ 15:07:25 present+ 15:08:13 topic: socialwg updates / websub implementation report 15:08:48 aaronpk: websub: there are now two or more implementation imports for each role (pub/sub/hub) 15:09:16 i've been recently thinking about sticking HTTP signatures on top of WebSub subscription requests so the hub could attach a user identity to a callback URL (optionally) 15:09:35 Gargron: I'll add this as a topic to the agenda 15:09:58 aaronpk: no new reports from the implementations in the GNU social space, Mastodon would be really nice to get 15:09:58 s/Gargron:/Gargron,/ 15:09:59 it's not so much that a user identity is useful because a callback URL is instance-wide but because there is no other concept of "instance" than the domain field on a user, that's basically what i want to use it for 15:10:07 go to websub.rocks and test there 15:10:16 aaronpk: we need more reports to go further through the process 15:10:17 (that part cut out of me) 15:10:26 mastodon is on pubsubhubbub 0.4 15:10:52 Gargron, it should be the same, we worked hard to make sure any pubsubhubbub is websub compliant 15:11:02 for 0.4 anyway 15:11:17 aaronpk: about PuSH 0.4, it should be compatible with WebSub 15:11:19 i am pretty sure websub.rocks worked with mastodon since the very start 15:12:16 Gargron: I don't know how important it is, the application of this is limited. There's a spec called HTTP signatures, came up before related to ActivityPub 15:12:22 q+ 15:12:57 Gargron: we can reuse the keys to do salmon keys, and for access control. for WebSub subscriptions, it'd be nice to link the subscriptions to instances or servers 15:13:54 Gargron: domains are [side effects], ... domain whitelisting, blacklisting. somehow link a callback url to the instance it comes from. 15:14:10 cwebber2: if you're adding signatures via http signatures, it'd be backwards compatible 15:14:18 Gargron: it'd be an extra piece of information, and wouldn't be required 15:14:39 aaronpk: why isn't the URL itself enough to use? the domain is part of the callback url, right? 15:15:18 Gargron: not needed, it can be on other domains, etc, be forwarded, and instances can be on subdomains and have forwarded the webfinger from the root 15:15:54 Gargron: we only send private statuses to instances if their domain is approved. but the callback url can be different. Domains that have a different host don't receive the private statuses 15:15:57 q+ 15:15:58 q? 15:16:05 ack puckipedia 15:16:09 scribenick: cwebber2 15:17:06 puckipedia: mostly in response to http signatures; I recently implemented server to server authorization on Koreg, which uses json web tokens with the key behind it. And I think it might be a slightly better option because it's already in RFC, whereas http signatures might expire. As for federation of websub urls, it sounds a bit hacky to use a random identity, for example a first follower 15:18:21 Gargron: yes you're right, it's hacky, but that's because this hasn't been a use case in the original protocol designs. They didn't consider an instance (user?) to be an entity by itself and have any agency. Doesn't have to be a random user, can be the first user that followers, but if different user in the future there has to be another method because salmon, etc. Re: JWT I dunno, but HTTP Signatures are dead simple to implement and 15:18:22 have that going for them 15:18:47 https://tools.ietf.org/html/draft-cavage-http-signatures-06 is the signature spec we're talking about right? 15:18:50 puckipedia: yeah, the way I solved that is not have a single centered subscription for the entire server, because that's mostly how activitypub works. so websub, ostatus support is kind of a side effect of the implementation 15:19:07 Gargron: wait that means if one server has one message, it has to send the message 20 times? 15:19:26 zatnosk has joined #social 15:19:41 puckipedia: yes, but also usually activitypub works on the other way, like you specifically determine which users to send it to, which can be a collection of all your followers, so the server pushes out the messages rather than a server subscribing to and endpoint that delivers the messages of that user 15:19:42 q+ 15:20:28 Gargron: yeah you're right, but it's still the same model.. the original server looks at the followers collection, then delivers to them. public endpoint though is shared by all users. That's a concern of mine though becaus if you scale that up that's a lot of http requests duplicated 15:20:50 puckipedia: yeah I just realized that doesn't work for websub subscriptions, so it doesn't work for that case. has to be addressed to the public collection for it to be used 15:21:21 Gargron: yes, in fact that's a bit problematic... assuming you have the same message, the server would still use the same info of who the users are. So... it's not very optimized, I dunno 15:21:23 q? 15:21:29 ack sandro 15:21:53 sandro: so two kind of different comments, this is cleanly a websub extension? we're not talking about a websub modification, this is an extension that the different components could use if they want to 15:21:54 scribenick: puckipedia 15:22:11 sandro: this is no reason to slow down websub, right? 15:22:17 Gargron: indeeds 15:23:01 sandro: I don't know http signatures very well, but this doesn't solve distribution. You still have to look up the key, can't you just check the hub url? 15:23:41 Gargron: not quite. the http signature contains the key id, algorithm, and signature (concatenation of a specific set of headers in an order). the key id can be anything. In case of mastodon, the key id is the account resource identifier 15:24:15 Gargron: which is most likely to be in the database. So you can just look it up in the database. if it isn't equal, you can look it up remotely 15:24:39 sandro: by the same mechanism of storing the public key, you could also store the hub url 15:25:06 sandro: your extension would be an add an 'on behalf of'. you could also add a webfinger field to where the subscriptions should be sent 15:25:32 Gargron: there's no connection between webfinger and signature url. The websub subscription contains a callback url, it's not in webfinger 15:26:19 Gargron: the hub server is supposed to validate the url. but, that's nothing to do with who's doing the subscription. and that's the key of what http signatures offers. it's not enough to say it's 'on behalf of'. what if a malicious user sends the request? That's what http signatures are for 15:26:55 sandro: would someone want to raise an issue to [explore how to solve this issue?] 15:27:10 https://github.com/swicg/general/issues 15:27:14 cwebber2: sounds like you just volunteered, sandro :) 15:27:24 q? 15:27:26 ack cwebber 15:28:01 cwebber2: so one thing I wanted to understand is slightly [tangental], the purpose of the publicInbox endpoint is to avoid the problem of where you have to do a lot of delivery to a lot of users 15:28:27 cwebber2: we did this in request of diaspora, do you think there are other cases that are not the publicInbox case that aren't handled by that delivery mechanism, that we're really missing? 15:29:01 Gargron: I understand that ActivityPub is very general, in the way it was created with the client API as well, it's oriented to writing messages to specific users 15:29:34 Gargron: in the case of adding activitypub, we won't get a whole new addressing. it's delivered to the followers and the people mentioned, in the case of public posts and private posts 15:30:13 Gargron: if I make a private post, would my server have to send 15000 requests to all servers? my view is that the publicInbox would be used for private posts too. the server would use its knowledge of the followers to do delivery to the specific individuals 15:30:14 q+ 15:30:41 cwebber2: my impression is where thing only would be used publicly isn't right, my issue is how we'd addrses this 15:30:50 cwebber2: you'd have to rely on the other server having a good sense of followers 15:31:07 ack puckipedia 15:31:12 scribenick: cwebber2 15:31:40 puckipedia: mostly a response to the delivery mechanism. Wouldn't it be possible to have a public inbox and also a list of people to deliver to on that server 15:32:03 Gargron: usually a server should keep track of followers, so there's already usually an internal follower list used for doing the home feeds of people, so seems like that info is already tracked 15:32:52 puckipedia: in the way that I implemented and think of how it should be? your inbox is basically your home feed, so every message that goes to followers of a person, you're addressed to that post. I don't really trust the collections on other servers, I don't use that information. I don't use that to determine if you follow someone. For example, if you block someone, it shouldn't be delivered to that person 15:33:26 Gargron: mastodon takes a slightly different approach, blocks are delivered as a salmon slap, but it's just to notify the server so it can update its own follower/following lists to prevent the user from seeing 15:33:28 a+ 15:33:30 q+ 15:33:37 scribenick: puckipedia 15:33:38 ack cwebber 15:34:29 cwebber2: so partly one way this might tie in is, I agree with puckipedia with that, specifically, the server it's delivering list, and I think a public inbox that also lists which people to deliver it to, instead of a server keepign track of who it should deliver to 15:35:30 cwebber2: you might not want to expose a block activity with a person. you may be federating a block list, without sending that to hostile people. another one, used as an unfortunate [??], there's been this conversation about 'advisory privacy' type things [?]. The challenge of ostatus private messages is because of a lack of explicitness 15:35:43 cwebber2: I worry that would go away if you post to a publicInbox and not specifying who it'd go to 15:36:06 Gargron: there's concerns about synchronizing follower lists, if e.g. a block has not been federated properly, and the person shouldn't be following the other anymore 15:36:46 Gargron: the main problem of the "leaky" privacy is the fact that the status of a message being private or not is bolted on, and not part of the protocol itself 15:37:12 Gargron: if the parser doesn't know what to do with the bolted on private message it defaults to public. this will be solved by ActivityPub, as public/private is part of the protocol 15:38:28 Gargron: the greatest approach would be to explicitly state which places a message would go to, and each place storing which users were targets. however, it's a problem of data modelling. storing a list of thousands of people that are unique for each individual message would be an explosion of storage and complexity. the way Mastodon models is that there are messages, followers, and your home feed is a sum of the messages of your followers 15:39:24 cwebber2: I was wondering if there was a publicInbox which has a list of audiences, would that be satisfactory? e.g. being explicit being a dealbreaker 15:40:24 Gargron: what would the remote server do with the explicit information? For notifications, we could store it. But messages haven't been made to be put in 12k people. as for home feeds, they're based on follower status. even if you get a list of targets, while it is possible to append it to the home feed in redis (cache layer), as soon as that expires 15:40:45 Gargron: e.g. being gone for 2 weeks, it has to be rebuilt based on the database. so those messages would be lost 15:42:07 cwebber2: I'm starting to understand the database side. Seeing how puckipedia and I implemented it, the inbox is a large collection, an ordered list you are tacking items onto. but my impression is, what you're saying, is that in MAstodon there's no such 'inbox' structure, you have each post with who its followers are, you build the inbox from that 15:42:19 cwebber2: so if you unfollow someone, old statuses disappear too? 15:42:43 tantek has joined #social 15:42:44 Gargron: there's two things: accounts, and statuses. Accounts follow other accounts, and are followed by others. this is how it's encoded into and from OStatus 15:42:54 Gargron: there is no concept of a polymorphic table you store the entire inbox into 15:43:37 Gargron: so yeah, that's mainly where the project comes from. The system wasn't build for this massive home feed for every user that stores each thing for each user 15:43:56 cwebber2: it'd be very complicated to calculate, right? would you think to moving a collection for each user would be maybe more efficient? 15:45:08 Gargron: it's pretty efficient. when a status appears, we trigger so-called FanOutOnWrite. it looks at the followers of the account who posted it (local or remote), for each follower we put the message on their home feed, the write-layer cache, a redis sorted set that contains the IDs of the statuses on each user online for the last 2 weeks 15:45:37 Gargron: the expensive query is when the feed is cleared out completely, when we have to look at all the messages of the people they're following 15:46:00 q? 15:46:02 cwebber2: I see why the DB structure incentivizes a certain pattern. I feel like I haven't completely thought it through. so, well, I don't know if I have anything else 15:47:18 Gargron: to reiterate, that system is kinda really efficient, when I was building Mastodon, I was researching how other systems implemented it. there's an article on a blog called something dot scalability. Twitter shared something about the architecture. the method I implemented is how Twitter does it as well, it's storage efficient and also very fast 15:47:29 Gargron: you can also do advanced filtering on the home feed 15:48:01 cwebber2: I suggest we stew on these structural differences and what it means and move on to further topics 15:48:11 q? 15:48:38 scribenick: puckipedia 15:48:50 scribenick: cwebber2 15:49:12 puckipedia: Kroeg is pronounced like "koough"(?) which is a Dutch pun on PubSub 15:49:23 (Pub) 15:49:27 topic: Kroeg implementation of ActivityPub 15:49:38 Gargron: may I ask how far you are in your implementation? 15:49:47 present+ 15:49:55 puckipedia: I think I've implemented everything, and I think I added useful extensions for clients. I added two extra endpoints 15:50:27 puckipedia: the first endpoint is a settings endpoint to get a url from where you can do changes to settings, eg changing passwords and etc. I've moved it into an endpoint that's basically a web page 15:50:46 cwebber: why not just put it on the user's profile? 15:51:13 puckipedia: because endpoints are public urls, and you can't get browser to explicitly use a specific authorization header... so in our case it gives the user 10 minutes of time to change settings 15:51:51 puckipedia: the other endpoint is basically all your relevant objects relevant to other objects... eg all the objects that are attributed to you which also have as object the (?) so you can get all the Likes and Announces of a specific post 15:52:03 puckipedia: it should also probably show all the Undos of those objects but that may come later 15:52:32 puckipedia: so a client could use that to show that you've already liked a specific object, so you can undo that like, mainly because AP has undos of everything instead of unlikes, etc 15:53:06 puckipedia: oh, and I added a list of blocked users to a user object, so you can look back at what things are blocked, so you can't see in the spec how that should work, so that's kind of interesting 15:53:51 puckipedia: it doesn't have any special features yet, but I'd like to add things like deferred authorization to specific servers, you can not tell... 15:54:18 cwebber: Any lessons learned from doing the implementation? 15:55:08 puckipedia: one weird thing I had was side effects of specific objects... eg user A of server A sends Like to server B on server B, I accidentally handled side effects twice because it was delivered multiple times. So I limited it to only when the user is notified that it gets the side effects 15:55:35 tantek: i am pretty sure it just works and you could try it yourself with someone's public profile 15:55:50 puckipedia: basically, when you follow a user but you do not send that activity to that user you're following, you will not be added to that list of followers, even if you are on the same server but not following that person 15:56:05 Gargron: always better to get a report done by someone *outside* the WG 15:56:54 Gargron: would you be ok with me filing an issue requesting this? That way maybe someone in the Mastodon community could pick it up? 15:58:32 Gargron: so I'm trying to make progress in AP in mastodon as well, and the way I'm doing this is adding the json representations of objects first, and it turns out it's kind of complicated. the pull request has been WIP for like 20 days, which is pretty long. some of the roadblocks are eg sensitive content, which I've added a boolean property, and the Tag type, which I've just added a Tag type. What's still problematic is which 15:58:32 internal visibility levels should correspond to to/cc objects. It's not clear how to represent that. It's implementation specific but I'd like help/review. But one thing is when to show Create of the object or just the Object. And this again has to do with the internal storage; there is basically a url structure where you basically retrieve a status. So a status can actually be something like an announce of another note, so either 15:58:32 I have to return Notes from the url structure except when it's reblog, in which case i return an announce, or I start to do Create activities for all of them and Announce for some others 15:58:46 Gargron: which one is the Create, which one is the object, which is the outer or inner? 15:58:58 {type: Announce, id: https://b.example.com/b#object, object: https://a.example.com/a#object} 15:58:58 Gargron: so yeah, that's been the main problem. the pull request is on github if you want to review/comment 15:58:59 q+ 15:59:10 puckipedia: ok so that's quite a bunch 15:59:16 scribenick: puckipedia 15:59:22 ack cwebber 16:00:04 cwebber2: so one thing I wanted to say is that this challenge of Creates being kind-of as .. activitypub, as a spec comes from pump.io, which had separate create objects. when it started to turn into ActivityPub it was called ActivityPump, written by oshepherd, familiar with the history of this space 16:00:18 :100: 16:00:21 cwebber2: originally, they wanted to remove Create, because it was very artificial. if it's just an object, you may as well ship it 16:00:59 cwebber2: the problem is that not everybody agreed with it (at least Evin[?]), you should always have the verb of the Activity, and when you look at someone's feed you should always see activity, some kind of subject, and some kind of object 16:01:02 s/subject/actor/ 16:01:32 I would 16:01:34 we dropped verbs in AS2 by consensus in SocialWG years ago as one of the first simplifications like in 2013? 2014? I'll see if I can look it up 16:01:35 cwebber2: so that's the reason the Create activity is there. We talked about this, because webmention doesn't do anything like this. we had talked about removing it, but we got strong pushback from Evin, and from my own implementaion, I wouldn't mind dropping it 16:02:01 we resolved to keep it in, and added the whole goofy mechanism that if you drop an object in an outbox it'd be wrapped in a Create activity. some history 16:02:06 s/Evin/Evan 16:02:22 so outbox should contain create/announce. the IDs would correspond to local statuses, and the status URIs would return the create/announce as well, and nothing would return the notes standalone - would that be fine? 16:02:26 cwebber2: not as short as I implied it was going to be. I feel your pain on the create wrapping stuff. if it was just me, we would make it optional. maybe we *should* make it optional, if there's enough implementors 16:02:40 scribenick: cwebber2 16:03:53 puckipedia: I think that the Create is actually pretty useful in some cases, because you get an outbox that is just activities, and each activity contains an object. that kind of IMO makes more sense, because you've got Create, Updates and Deletes. I think if just posting objects I think it would be less clear. I don't think an object is required to have any attributedTo or anything, so I think the actor is the only official way to 16:03:53 make the object 16:04:06 Gargron: I agree that without the announce activity I think it would be hard to tell how an object was there 16:04:19 Gargron: in the outbox I definitely agree Create/Announce activities should be there 16:05:24 Gargron: but they both currently require an id, and it's not clear that both should be resolvable. but both id should resolve to the actual id of the object, so the question actually becomes when the corresponding user has a unique identifier, if queried by itself should it show the Create/Announce or the actual object. the consistent way its the outbox has the Create/Announce, and if you pull back the Create/Announce and nothing has 16:05:24 the Note itself 16:05:42 Gargron: but on the other hand what do you do when you have an Announce and instead of the object being embedded, you have only the URI of the thing being announced 16:06:02 Gargron: if you go to the id it should give the Create, that would be inconsistent 16:07:04 puckipedia: so there are multiple ways to fix this, for example one of the ways you can do it is by having always a url which is the url of the Create activity or the Announce activity, and always have the id inside, and the fragment is inside, you can get the Create around it and store that too, returning only the Note it contains, so you could have a fragment identifier or have a sub-id 16:07:17 puckipedia: as far as I can tell from the spec any object with an id should be resolvable 16:07:18 q+ 16:07:25 puckipedia: so any Create should be resolvable separately 16:07:28 so.... activity = real id. object = fragment id inside the activity 16:07:37 scribenick: puckipedia 16:07:41 ack cwebber2 16:07:43 ack cwebber 16:07:47 if that's what you mean that could work, makes sense 16:08:23 cwebber2: fragment identifiers are tricky, because there's no way it can be resolved. Kroeg assumes you can find the top-level object and look through it. that's nice, but not guaranteed 16:08:48 cwebber2: fragment identifiers are best used where you have no guarantee you can retrieve it independently. so I would advice each object having a full-fledged identifier 16:09:00 cwebber2: one solution is, I'm going to type it up on IRC 16:09:03 https://mastodon.example/post/123 <- object 16:09:14 https://mastodon.example/post/123/create <- "artificially generated" Create object 16:09:53 cwebber2: so you cna have your base object, and since Mastodon is generating things from its highly structured database, so you can generate a Create object for that, that's dynamically generated. that'd be the cleanest result for what I understand is Mastodon's design 16:10:19 Wait. How does that handle user generated creates if it's dynamically created on the server? 16:10:24 I am probably misunderstanding 16:10:33 cwebber2: I kind of like fragment identifiers, and I kind of hate them. there's one person here who hates this usage, it can be a bit tricky. If you want people to be able to resolve it the best way is to give it a separate url 16:10:59 cwebber2: answering tsyesika; say, you had an object like this 16:11:04 {"type": "Note", "content": "I'm a note", "author": } 16:11:29 {"type": "Create", "object": {"type": "Note", "content": "I'm a note", "author": }} 16:11:29 cwebber2: and if you'd visit /create, it'd be wrapped into something like this: 16:11:51 cwebber2: I think that's the cleanest design for Mastodon's case, I wonder how you feel about that, Gargron 16:12:16 Gargron: I think that would probably work, didn't [..] want to add additional routes, but that's going to work, it's not the worst thing 16:12:37 But this is to prevent having to keep creates in the db right? What if the user makes the create and submits that via the client API? 16:13:19 Gargron: in case of the /announce, the url without the /announce would be the remote one 16:13:34 ah I see. I'm with you 16:14:04 I wonder how it'd store remote creates federated them 16:14:05 {"id": "https://localmasto.example/post/8351", "type": "Announce", "object": {"id": "https://remotemasto.example/post/515/", "type": "Note", "content": "I'm a note"}} 16:15:35 puckipedia: chris is 16:16:57 cwebber2: my suggestion of the artificial create would be only for the Creates, but for the others they wouldn't have to be dynamically generated. I understand your feeling of ickyness for the particular url structure 16:17:07 yeah sorry about making this discussion too implementation specific 16:17:36 [sandhawke] #5 In-Page Social Interactivity 16:17:53 Gargron: its okay, implementation info is always good 16:17:57 thanks cwebber2 16:18:08 trackbot, end meeting 16:18:08 Zakim, list attendees 16:18:08 As of this point the attendees have been tsyesika, aaronpk, cwebber, puckipedia, ben_thatmustbeme, Gargron, tantek 16:18:16 RRSAgent, please draft minutes 16:18:16 I have made the request to generate http://www.w3.org/2017/07/12-social-minutes.html trackbot 16:18:17 RRSAgent, bye 16:18:17 I see no action items